Trajectory generation method, system, device and electronic equipment

Abstract

This invention provides a trajectory generation method, system, apparatus, and electronic device, applied in the field of intelligent vision technology. The method, applied to a management device in a trajectory generation system, includes: receiving object information generated by each camera for a captured object, the object information including: the trajectory and feature information of the object within the camera's field of view; for each trajectory within the camera's field of view, identifying the identity information of the object to which the trajectory belongs based on the feature information corresponding to the trajectory, using this as the identity information corresponding to the trajectory; based on each trajectory, and the feature information and identity information corresponding to each trajectory, determining trajectories belonging to the same object from the trajectories within the field of view of each camera; and for each object, fusing the trajectories belonging to that object to obtain the trajectory of that object in the target scene. This solution enables seamless generation of object trajectories.

Description

Technical Field

[0001] This invention relates to the field of intelligent vision technology, and in particular to trajectory generation methods, systems, devices and electronic devices. Background Technology

[0002] The analysis of object trajectories is the foundation for applications with significant commercial value, such as store promotion, target scenario planning, and advertising bidding. It is also an indispensable part of public safety and emergency response plans, and has received increasing attention from researchers in recent years.

[0003] In related technologies, RFID (Radio Frequency Identification) wristbands are often used to generate the trajectory of an object indoors. Simply put, the object needs to wear an RFID wristband beforehand, and the position of the RFID wristband is obtained as the object moves indoors, thereby determining the object's trajectory.

[0004] Because the relevant technology requires the object to wear an RFID wristband, it is impossible to generate the object's trajectory seamlessly. Summary of the Invention

[0005] The purpose of this invention is to provide a trajectory generation method, system, device, and electronic device to achieve seamless trajectory generation of objects. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of the present invention provide a trajectory generation method applied to a management device in a trajectory generation system, the trajectory generation system further comprising: multiple cameras deployed in a target scene, the method comprising:

[0007] Receive object information generated by each camera for the captured object. The object information of each camera includes: the trajectory and feature information of the object within the field of view of that camera;

[0008] For the trajectory of an object within the field of view of each camera, the identity information of the object to which the trajectory belongs is identified based on the feature information corresponding to the trajectory, and used as the identity information corresponding to the trajectory; wherein the feature information corresponding to each trajectory is: the feature information of the object to which the trajectory belongs;

[0009] Based on each trajectory, and the feature information and identity information corresponding to each trajectory, each trajectory belonging to the same object is determined from the trajectories within the field of view of each camera; for each object, the trajectories belonging to that object are fused to obtain the trajectory of that object in the target scene.

[0010] Optionally, the step of determining the trajectories belonging to the same object from the trajectories within the field of view of each camera, based on each trajectory and the corresponding feature information and identity information, includes:

[0011] The trajectory within the field of view of each camera, along with the feature information and identity information corresponding to each trajectory, are input into a pre-trained trajectory association model. The trajectory association model is then used to determine the trajectories belonging to the same object from the trajectories within the field of view of each camera.

[0012] The trajectory association model is trained using multiple sample trajectories, the feature information and identity information corresponding to each sample trajectory, and the calibration information of each sample trajectory. The sample calibration information of each sample trajectory indicates the sample trajectory that belongs to the same object as the sample trajectory.

[0013] Optionally, after performing trajectory fusion on the trajectories belonging to the object to obtain the object's trajectory in the target scene, the method further includes:

[0014] For each object, the object's identity information is determined based on the identity information corresponding to the trajectory within the field of view of each camera belonging to that object, and is used as the identity information associated with the object's trajectory in the target scene.

[0015] Optionally, determining the object's identity information based on the identity information corresponding to the trajectory within the field of view of each camera belonging to the object includes:

[0016] From the identity information corresponding to the trajectory within the field of view of each camera belonging to the object, select the identity information with the highest proportion as the identity information of the object.

[0017] Secondly, embodiments of the present invention provide a trajectory generation system, the trajectory generation system including a management device and multiple cameras deployed in a target scene, wherein:

[0018] A target camera is used to generate object information of the captured object; wherein, the object information of the captured object includes: the trajectory and feature information of the captured object within the field of view of the target camera; the target camera is any camera in the trajectory generation system;

[0019] The management device is configured to, after receiving object information sent by each camera in the trajectory generation system, identify the identity information of the object to which the trajectory belongs based on the feature information corresponding to the trajectory within the field of view of each camera, and use this identity information as the identity information corresponding to the trajectory; based on each trajectory and the feature information and identity information corresponding to each trajectory, determine each trajectory belonging to the same object from the trajectories within the field of view of each camera; and for each object, perform trajectory fusion on each trajectory belonging to the object to obtain the trajectory of the object in the target scene; wherein the feature information corresponding to each trajectory is: the feature information of the object to which the trajectory belongs.

[0020] Optionally, the target camera generates the trajectory of objects within its field of view in the following manner:

[0021] Record the position information of objects within the field of view according to the specified location acquisition period;

[0022] For each object within the field of view, the trajectory of the object within the camera's field of view is generated based on the position information of the object recorded at each location.

[0023] Optionally, the target camera is specifically used to perform object detection on the acquired images; when an object is detected in the image, the position information of the detected object is recorded at each position acquisition time according to a specified position acquisition period.

[0024] Optionally, the target camera is specifically used to perform face detection, head and shoulder detection, and human body detection on the acquired images; when a face, head and shoulder, and human body are detected, the detected face, head and shoulder, and human body are associated; if there are associated faces, heads and shoulders, and human bodies, the associated faces, heads and shoulders, and human bodies are used as the detected objects.

[0025] Optionally, the location information is: the first coordinate of the object in the camera coordinate system;

[0026] The target camera is specifically used to generate a first coordinate linked list of the object according to the order of the location acquisition times, based on the first coordinate of the object recorded at each location acquisition time; based on the mapping relationship between the camera coordinate system and the scene coordinate system of the target scene, each first coordinate in the location information linked list is converted into a second coordinate in the scene coordinate system to obtain a second coordinate linked list, which serves as the trajectory of the object within the field of view of the camera.

[0027] Optionally, the target camera generates feature information of objects within its field of view in the following manner:

[0028] At each location, an object image of the object within the camera's field of view is acquired. For each object within the camera's field of view, attribute information is extracted from each object image of the object. Based on the attribute information in each object image of the object, object images whose attribute information conforms to preset attribute filtering rules are selected from each object image of the object. Each selected object image and its attribute information are used as the feature information of the object to which the object image belongs.

[0029] Optionally, the object image includes: a face image and / or a human body image;

[0030] The target camera is specifically used to extract facial attribute information for each face image of the object; and / or, extract human attribute information for each human body image of the object; filter out face images whose facial attribute information conforms to preset facial attribute filtering rules from each face image of the object; and / or, filter out human body images whose human attribute information conforms to preset human attribute filtering rules from each human body image of the object.

[0031] Optionally, the management device is specifically used to input the trajectory within the field of view of each camera, as well as the feature information and identity information corresponding to each trajectory, into a pre-trained trajectory association model, so as to use the trajectory association model to determine the trajectories belonging to the same object from the trajectories within the field of view of each camera; wherein, the trajectory association model is trained using multiple sample trajectories, the feature information and identity information corresponding to each sample trajectory, and the calibration information of each sample trajectory, and the sample calibration information of each sample trajectory indicates the sample trajectory belonging to the same object as the sample trajectory.

[0032] Optionally, the management device is further configured to, after performing trajectory fusion on each trajectory belonging to the object to obtain the trajectory of the object in the target scene, determine the identity information of the object for each object based on the identity information corresponding to the trajectory within the field of view of each camera belonging to the object, and use this as the identity information associated with the trajectory of the object in the target scene.

[0033] Optionally, the management device is specifically used to select the identity information with the highest proportion from the identity information corresponding to the trajectory within the field of view of each camera belonging to the object, and use it as the identity information of the object.

[0034] Thirdly, embodiments of the present invention provide a trajectory generation device, applied to a management device in a trajectory generation system, wherein the trajectory generation system further includes: multiple cameras deployed in a target scene, and the device includes:

[0035] The information receiving module is used to receive object information generated by each camera for the captured object. The object information of each camera includes: the trajectory and feature information of the object within the field of view of the camera.

[0036] The information recognition module is used to identify the identity information of the object to which the trajectory belongs based on the feature information corresponding to the trajectory for each object within the field of view of the camera, and use this as the identity information corresponding to the trajectory; wherein the feature information corresponding to each trajectory is: the feature information of the object to which the trajectory belongs;

[0037] The trajectory determination module is used to determine the trajectories belonging to the same object from the trajectories within the field of view of each camera, based on each trajectory and the feature information and identity information corresponding to each trajectory.

[0038] The trajectory fusion module is used to fuse the trajectories belonging to each object to obtain the trajectory of the object in the target scene.

[0039] Optionally, the trajectory determination module is specifically used to input the trajectories within the field of view of each camera, as well as the feature information and identity information corresponding to each trajectory, into a pre-trained trajectory association model, so as to use the trajectory association model to determine the trajectories belonging to the same object from the trajectories within the field of view of each camera; wherein, the trajectory association model is trained using multiple sample trajectories, the feature information and identity information corresponding to each sample trajectory, and the calibration information of each sample trajectory, and the sample calibration information of each sample trajectory indicates the sample trajectory belonging to the same object as the sample trajectory.

[0040] Optionally, the device further includes:

[0041] The identity determination module is used to determine the identity information of each object based on the identity information corresponding to the trajectory within the field of view of each camera belonging to the object after the trajectory fusion module performs trajectory fusion on each trajectory belonging to the object to obtain the trajectory of the object in the target scene. This identity information is used as the identity information associated with the trajectory of the object in the target scene.

[0042] Optionally, the identity determination module is specifically used to select the identity information with the highest proportion from the identity information corresponding to the trajectory within the field of view of each camera belonging to the object, and use it as the identity information of the object.

[0043] Fourthly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0044] Memory, used to store computer programs;

[0045] When a processor executes a program stored in memory, it implements any of the steps described in the first aspect.

[0046] Fifthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any of the first aspects.

[0047] Beneficial effects of the embodiments of the present invention:

[0048] In the trajectory generation method provided by the embodiments of the present invention, since the trajectory and feature information generated by each camera for the captured object can be received, and then based on the trajectory and feature information of each object, the trajectories belonging to the same object are determined from the trajectories within the field of view of each camera, and the trajectory of each object in the target scene is obtained through trajectory fusion, so that the trajectory of the object can be generated by the camera alone, realizing the seamless generation of the object's trajectory.

[0049] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a schematic diagram of the trajectory generation system provided in an embodiment of the present invention;

[0052] Figure 2 This is a flowchart of a trajectory generation method provided by an embodiment of the present invention from the perspective of a management device;

[0053] Figure 3 This is another flowchart of the trajectory generation method provided by an embodiment of the present invention from the perspective of management equipment;

[0054] Figure 4 This is a schematic diagram of the trajectory generation system provided in an embodiment of the present invention;

[0055] Figure 5 This is a flowchart illustrating the process of generating object trajectories using a target camera in an embodiment of the present invention.

[0056] Figure 6This is a flowchart illustrating the feature information of the target camera in an embodiment of the present invention.

[0057] Figure 7 This is a schematic diagram of the trajectory generation device provided by an embodiment of the present invention from the perspective of management equipment;

[0058] Figure 8 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] To achieve seamless generation of object trajectories, embodiments of the present invention provide a trajectory generation method, system, device, and electronic device.

[0061] like Figure 1 The diagram shown is a structural schematic of a trajectory generation system provided in an embodiment of the present invention. The trajectory generation system includes a management device and multiple cameras (such as...). Figure 1 The system includes cameras 1 to N, where multiple cameras are deployed in the target scene. The management device can be deployed either inside or outside the target scene as needed. This embodiment of the invention does not impose specific limitations on the deployment location of the management device, as long as it can communicate with each camera in the target scene.

[0062] For example, the target scenario is an office area containing multiple offices, with at least one camera installed in each office. This ensures that the entire office area is covered by the cameras' field of view. Furthermore, each camera is connected to a management device via wired or wireless means. This management device can be installed in any office within the office area, in an observation room outside the office area, or deployed in a remote backend.

[0063] Each camera in the target scene can collect video data of the observation area in real time. When a specified type of object appears in the observation area of ​​any camera, the object can be observed. After the object leaves the observation area, the object information of the object can be generated based on the object's activity information within the camera's field of view. The object information of the object can then be sent to the management device in real time or periodically.

[0064] The management device can receive object information sent by each camera in real time or periodically. It can then combine this information to determine the object's complete movement trajectory within the target area and identify the object's identity. The specific process will be described in detail in subsequent embodiments and will not be repeated here.

[0065] It should be noted that the target scenarios mentioned above can include hotels, shopping malls, supermarkets, office areas, etc. The management equipment mentioned above can be various electronic devices, such as personal computers, servers, mobile phones, and other devices with data processing capabilities. The camera mentioned above can be a smart camera with data processing capabilities, or a combination of a camera with data acquisition capabilities and a device with data processing capabilities; all of these are possible. The trajectory generation method provided in this embodiment of the invention can be implemented through software, hardware, or a combination of both.

[0066] The following describes a trajectory generation method provided by an embodiment of the present invention from the perspective of the management device in the trajectory generation system.

[0067] The trajectory generation method provided in this embodiment of the invention may include the following steps:

[0068] Acquire object information generated by each camera for the captured object. The object information includes the trajectory and feature information of the object within the camera's field of view.

[0069] For the trajectory of an object within the field of view of each camera, the identity information of the object to which the trajectory belongs is identified based on the feature information corresponding to the trajectory, and used as the identity information corresponding to the trajectory; wherein the feature information corresponding to each trajectory is: the feature information of the object to which the trajectory belongs;

[0070] Based on each trajectory, and the corresponding feature and identity information, each trajectory belonging to the same object is determined from the trajectories within the field of view of each camera; for each object, the trajectories belonging to that object are fused to obtain the trajectory of that object in the target scene.

[0071] In the above-described solution provided by the embodiments of the present invention, since the trajectory and feature information generated by each camera for the captured object can be obtained, and then based on the trajectory and feature information of each object, the trajectories belonging to the same object are determined from the trajectories within the field of view of each camera, and the trajectory of each object in the target scene is obtained through trajectory fusion, thus the trajectory of the object can be generated by the camera alone, realizing the seamless generation of the object's trajectory.

[0072] The trajectory generation method provided by the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0073] like Figure 2As shown, this embodiment of the invention provides a trajectory generation method applied to a management device in a trajectory generation system. The trajectory generation system further includes multiple cameras deployed in a target scene, and may include the following steps:

[0074] S201, Receive object information generated by each camera for the captured object, the object information including: the trajectory and feature information of the object within the camera's field of view;

[0075] Depend on Figure 1 As shown in the schematic diagram of the trajectory generation system, the management device establishes communication with each camera in the target scene. Therefore, after any camera generates object information for an object, it can send that object information to the management device, which can receive the object information sent by each camera. It should be emphasized that this embodiment of the invention does not specifically limit the communication method between the camera and the management device; for example, wired communication, wireless communication, real-time communication, and periodic communication are all possible.

[0076] In this embodiment of the invention, for any camera, when an object is captured at a certain distance, the camera can acquire the object's position information in real time within its field of view, thereby generating the object's trajectory within its field of view. Furthermore, based on the captured image of the object, the camera can identify the object's feature information. After the object leaves its field of view, object information can be generated and sent to the management device. It should be noted that the specific method by which any camera generates object information will be described in detail in subsequent embodiments and will not be repeated here.

[0077] In this context, the trajectory of the object captured by each camera is the movement path of the object within the field of view of the camera. For example, it can be the position information of the object at each moment when it is within the field of view of the camera. In one example, at time t1, the position information of object a1 on camera c1 is (Xt1,Yt1), then the trajectory of object a1 generated by camera c1 contains {t1,(Xt1,Yt1)}.

[0078] The feature information in the object information captured by each camera includes: information characterizing the information generated by the object, such as the image information of the object, and / or identity features extracted based on the image information of the object. In one example, if the object is a person, the object information may include at least one of the following: face image, head shape, hairstyle, hair color, whether a mask is worn, whether glasses are worn, gender, face orientation, body image, clothing color, clothing category, whether the body is occluded, or the body orientation. The above object information may also include evaluation information of the feature information, which can be information obtained by evaluating the feature information according to preset evaluation rules. The evaluation information is used to distinguish the feature quality of the feature information. For example, if the feature information is a face image, the evaluation information can score the face image based on whether the face is occluded, the proportion of the occluded area, the face orientation, and the clarity, thus obtaining a score for the face image.

[0079] S202, for the trajectory of an object within the field of view of each camera, based on the feature information corresponding to the trajectory, identify the identity information of the object to which the trajectory belongs, and use it as the identity information corresponding to the trajectory; wherein the feature information corresponding to each trajectory is: the feature information of the object to which the trajectory belongs;

[0080] In order to identify the identity information of the object to which each trajectory belongs, a correspondence between feature information and identity information can be pre-constructed. Then, for the trajectory of each object within the field of view of the camera, the identity information corresponding to the feature information of the trajectory can be determined based on the pre-constructed correspondence between feature information and identity information, and used as the identity information of the object to which the trajectory belongs.

[0081] For example, the pre-constructed correspondence between feature information and identity information includes: feature information d1 - identity information s1, feature information d2 - identity information s2, and feature information d3 - identity information s3. The object information of object a2 within the field of view of camera c2 includes trajectory g1 and feature information d1. Based on the correspondence between feature information and identity information, it can be determined that the identity information of trajectory g1, corresponding to feature information d1 and identity information s1, is identity information s1.

[0082] The following example, using a person as the subject and facial images as the feature information, further illustrates this step.

[0083] To identify the identity information of the person to whom each trajectory belongs, a facial database can be built for the people entering the target scene. This database contains facial images of each person entering the target scene and their corresponding identity information. Then, the management device can determine a second facial image that matches the first facial image in the facial database based on the first facial image of the person within the field of view of the first camera. Furthermore, it can determine the identity information corresponding to the second facial image as the identity information of the first facial image, and thus use it as the identity information of the trajectory of the person to whom the first facial image belongs within the field of view of the first camera.

[0084] The aforementioned identity information can be information that can uniquely identify an object, such as an Identity Document (ID).

[0085] S203, based on each trajectory and the feature information and identity information corresponding to each trajectory, determine the trajectories belonging to the same object from the trajectories within the field of view of each camera;

[0086] Since each camera's field of view may only cover a portion of the target scene, when an object moves within the target scene, it moves within the field of view of multiple cameras. Therefore, each of these cameras can generate the object's trajectory. For example, if cameras c1, c2, and c3 are deployed within the target scene, the target object first moves within the field of view of camera c1, then into the field of view of camera c2, and finally into the field of view of camera c3. The object information received by the management device includes the target object's trajectory g1 in the object information of camera c1, g2 in the object information of camera c2, and g3 in the object information of camera c3.

[0087] To determine the complete movement trajectory of an object within the target location, it is necessary to identify the trajectories belonging to the same object from the trajectories within the field of view of each camera. Since the movement trajectory of the same object within the field of view of each camera exhibits temporal and spatial continuity, and the feature and identity information of the same object should be identical across all cameras, it is possible to identify the individual trajectories belonging to the same object from the trajectories within the field of view of each camera based on each trajectory and the corresponding feature and identity information.

[0088] In one implementation, a trajectory association model can be pre-trained using multiple sample trajectories, the feature information and identity information corresponding to each sample trajectory, and the calibration information of each sample trajectory. This trajectory association model can then determine the trajectories belonging to the same object from the trajectories within the field of view of each camera. The sample calibration information of each sample trajectory indicates the sample trajectory belonging to the same object.

[0089] After obtaining the trained trajectory association model, the trajectories within the field of view of each camera, as well as the feature information and identity information corresponding to each trajectory, can be input into the pre-trained trajectory association model. The trajectory association model can then be used to determine the trajectories belonging to the same object from the trajectories within the field of view of each camera.

[0090] The aforementioned trajectory association model can be a model obtained by training a machine learning classification algorithm. It trains the model by using multiple sample trajectories, the feature information and identity information corresponding to each sample trajectory, and the calibration information of each sample trajectory, and then uses the algorithm with the trained parameters as the trajectory association model after training is completed.

[0091] A well-trained trajectory association model can be used to determine whether any two trajectories belong to the same object. Thus, after inputting each trajectory into the trajectory association model, the model can output the probability that any two trajectories belong to the same object.

[0092] It is important to emphasize that, in order to reduce the impact of noise on the output of the trajectory association model, each trajectory can be improved before being input into the trajectory association model. For example, for any trajectory, it can be processed by removing trajectory noise points, removing trajectory interruptions, or performing trajectory frequency fixing.

[0093] S204. For each object, perform trajectory fusion on each trajectory belonging to that object to obtain the trajectory of that object in the target scene.

[0094] Optionally, before performing trajectory fusion on trajectories belonging to a single object, the trajectories belonging to the same object can be grouped into trajectory groups, with each trajectory group containing trajectories belonging to the same object. After obtaining the trajectory groups, trajectory fusion can be performed on the trajectories within each trajectory group.

[0095] Furthermore, by combining the location information of each physical object in the target scene, obstacle correction and path interpolation can be performed on the obtained fused trajectory to solve the problems of trajectory positioning drift to non-co-travel areas or trajectory breakage caused by unsatisfactory correlation effects.

[0096] Since the fusion process may involve overlapping coverage areas of multiple cameras or trajectory fusion failures, resulting in inconsistent frame rates in the generated trajectories, or fusion errors may occur during the fusion process, causing offsets in the fused trajectories, further frequency fixing and smoothing operations can be performed on the trajectories.

[0097] In the above-described solution provided by the embodiments of the present invention, since the trajectory and feature information generated by each camera for the captured object can be received, and then based on the trajectory and feature information of each object, the trajectories belonging to the same object are determined from the trajectories within the field of view of each camera, and the trajectory of each object in the target scene is obtained through trajectory fusion, so that the trajectory of the object can be generated by the camera alone, realizing the seamless generation of the object's trajectory.

[0098] based on Figure 2 As shown in the embodiment, this invention also provides a trajectory generation method for a management device applied in a trajectory generation system, such as... Figure 3 As shown, after S204 above, the method further includes step S205:

[0099] S205, for each object, based on the identity information corresponding to the trajectory within the field of view of each camera belonging to that object, determine the identity information of that object as the identity information associated with the trajectory of that object in the target scene.

[0100] Because in the aforementioned process, there are cases where the identity information corresponding to the trajectory is incorrectly confirmed but the fused trajectory is correct, resulting in multiple identity information in the multiple trajectories contained in the fused trajectory, it is necessary to unify the identity of the fused trajectory, that is, to select a unique identity information as the information of the final fused trajectory.

[0101] In this step, for each object, the object's identity information can be determined based on the identity information corresponding to the trajectory within the field of view of each camera belonging to that object, and this identity information is used to associate the object's trajectory in the target scene.

[0102] In one implementation, for each object, the identity information with the highest proportion can be selected from the identity information corresponding to the trajectory within the field of view of each camera belonging to that object, and used as the identity information of that object.

[0103] For example, the fused trajectory G is obtained by fusing trajectory g1, trajectory g2 and trajectory g3. The identity information of trajectory g1 is humanid-a, the identity information of trajectory g2 is humanid-a, and the identity information of trajectory g3 is humanid-b. Since humanid-a accounts for the largest proportion, the identity information of the fused trajectory G is humanid-a, that is, the identity information of the object is humanid-a.

[0104] In the above-described solution provided by this invention, since the trajectory and feature information generated by each camera for the captured object can be received, and then based on the trajectory and feature information of each object, the trajectories belonging to the same object are determined from the trajectories within the field of view of each camera, and the trajectory of each object in the target scene is obtained through trajectory fusion, thus realizing the trajectory generation of the object using only the camera, achieving seamless generation of the object's trajectory. Furthermore, the identity information of the object corresponding to the fused trajectory can be determined, completing the unification of identity information and trajectory.

[0105] Based on the above method, embodiments of the present invention also provide a trajectory generation system. For example... Figure 4 As shown in the embodiment of the present invention, a trajectory generation system includes a management device 402 and multiple cameras 401 deployed in a target scene, wherein:

[0106] The target camera 401 is used to generate object information of the captured object; wherein, the object information of the captured object includes: the trajectory and feature information of the object captured within the field of view of the target camera; the target camera is any camera in the trajectory generation system;

[0107] The management device 402 is used to, after receiving object information sent by each camera in the trajectory generation system, identify the identity information of the object to which the trajectory belongs based on the feature information corresponding to the trajectory within the field of view of each camera, and use this identity information as the identity information corresponding to the trajectory; based on each trajectory and the feature information and identity information corresponding to each trajectory, determine each trajectory belonging to the same object from the trajectories within the field of view of each camera; for each object, perform trajectory fusion on each trajectory belonging to the object to obtain the trajectory of the object in the target scene; wherein the feature information corresponding to each trajectory is: the feature information of the object to which the trajectory belongs.

[0108] In the above-described solution provided by this invention, since the trajectory and feature information generated by each camera for the captured object can be received, and then based on the trajectory and feature information of each object, the trajectories belonging to the same object are determined from the trajectories within the field of view of each camera, and the trajectory of each object in the target scene is obtained through trajectory fusion, thus realizing the trajectory generation of the object using only the camera, achieving seamless generation of the object's trajectory. Furthermore, the identity information of the object corresponding to the fused trajectory can be determined, completing the unification of identity information and trajectory.

[0109] The present invention also provides a trajectory generation system, which is relatively simple to describe since it corresponds to the method provided from the perspective of management equipment. For relevant details, please refer to the description of the method provided from the perspective of management equipment.

[0110] Optionally, in one embodiment, such as Figure 5 As shown, the target camera generates the trajectory of objects within its field of view in the following manner:

[0111] S501, according to the specified position acquisition period, records the position information of objects within the field of view of the target camera;

[0112] The specified location acquisition period can be the camera's image acquisition period, meaning that the location information of the object within the image is recorded once for each acquired image. Alternatively, to reduce computational load, the specified location acquisition period can be N times the camera's image acquisition period, where N is an integer greater than 1, allowing the object's location information to be recorded every N images.

[0113] In one implementation, the aforementioned recording of the position information of the object within the field of view of the target camera may include steps 1-2:

[0114] Step 1: Perform object detection on the acquired images;

[0115] This includes object detection on the acquired images, which can include detection of parts of the object. For example, if the object is a person, human detection can include face detection, head and shoulder detection, and / or body detection on the acquired images.

[0116] When the object is a person, in order to improve the accuracy of object recognition, face detection, head and shoulder detection, and human body detection can be performed simultaneously.

[0117] When a face, head, shoulders, or body is detected, the detected face, head, shoulders, or body is associated with it; if an associated face, head, shoulders, or body exists, the associated face, head, shoulders, or body is used as the detected object.

[0118] In other words, after performing face detection, head and shoulder detection, and human body detection on the image, we can obtain the faces, heads and shoulders, and human bodies in the image. Then, we can use the position information of the detected faces, heads and shoulders, and human bodies in the image to associate them. Specifically, we can associate the closest faces, heads and shoulders, and human bodies, and then use the associated faces, heads and shoulders, and human bodies as the detected objects.

[0119] Step 2: When an object is detected in the acquired image, the position information of the detected object is recorded at each position according to the specified acquisition period.

[0120] After detecting an object, the position information of the detected object can be obtained in real time. Then, every time the duration of a specified position acquisition period elapses, the position information of the object is recorded once.

[0121] For example, at position acquisition time 1, the recorded position information is (x1, y1), and at position acquisition time 2, the recorded position information is (x2, y2).

[0122] S502. For each object within the field of view of the target camera, based on the position information of the object recorded at each position acquisition time, generate the trajectory of the object within the field of view of the target camera.

[0123] Among them, the position information of the object can be the first coordinates of the object in the camera coordinate system of the camera, such as (x1, y1) and (x2, y2) in the above example.

[0124] Furthermore, based on the position information of the object recorded at each position acquisition time, generating the trajectory of the object within the field of view of the target camera may include steps a - step b:

[0125] [[ID=tmp]]Step a: According to the chronological order of the position acquisition times, based on the first coordinates of the object recorded at each position acquisition time, generate a first coordinate linked list of the object;

[0126] Exemplarily, at acquisition time t1, the position information of the object is (x1, y1), denoted as <x1, y1, t1>, at acquisition time t2, the position information of the object is (x2, y2), denoted as <x2, y2, t2>, at acquisition time t3, the position information of the object is (x3, y3), denoted as <x3, y3, t3>, at acquisition time t4, the position information of the object is (x4, y4), denoted as <x4, y4, t4>. Among them, t1 < t2 < t3 < t4, then the generated first coordinate linked list is {<x1, y1, t1>, <x2, y2, t2>, <x3, y3, t3>, <x4, y4, t4>}.

[0127] Step b: Based on the preset mapping relationship between the camera coordinate system and the scene coordinate system of the target scene, map the first coordinates in the first coordinate linked list to the second coordinates in the scene coordinate system to obtain a second coordinate linked list as the trajectory of the object within the field of view of the target camera.

[0128] Among them, the mapping relationship between the camera coordinate system and the scene coordinates of the target scene can be, after the trajectory generation system is deployed, based on the position where each camera is deployed in the target scene and the field of view of each camera, determine the relationship between the coordinate system of each camera and the scene coordinate system of the target scene.

[0129] Once the first coordinate chain is determined, the mapping relationship between the camera coordinate system and the scene coordinate system of the target scene can be used to map each first coordinate in the first coordinate chain to the scene coordinate system of the target scene, thereby obtaining the mapped second coordinate and thus obtaining the second coordinate chain.

[0130] For example, the first coordinate chain is: {<x1,y1,t1> ,<x2,y2,t2> ,<x3,y3,t3> ,<x4,y4,t4> After mapping, the second coordinate chain is obtained as: {<x1’,y1’,t1’> ,<x2’,y2’,t2’> ,<x3’,y3’,t3’> ,<x4’,y4’,t4’>}

[0131] In the above-described solution provided by the embodiments of the present invention, the trajectory of an object can be generated using only a camera, achieving seamless generation of the object's trajectory. Furthermore, each camera can periodically record the position information of objects within its field of view at a specified location, thereby generating the trajectory of objects within the field of view of the target camera, thus providing a foundation for seamlessly generating the object's trajectory.

[0132] Optionally, in one embodiment, Figure 4 The target camera in the trajectory generation system shown is, for example... Figure 6 As shown, the following steps are used to generate feature information of objects within the field of view of the target camera:

[0133] S601, at each position acquisition time, acquires the object image of the object within the field of view of the target camera;

[0134] The aforementioned object image can be a complete view of the object or a partial view of the object; either is acceptable. Optionally, if the object is a person, the object image can include a view of the face and / or the body.

[0135] S602: For each object within the field of view of the target camera, extract the attribute information of each object image of that object, and based on the attribute information of each object image, filter out the object images whose attribute information conforms to the preset attribute filtering rules from the object images of that object.

[0136] When the object image includes face images and / or body images, for each object within the field of view of the target camera, facial attribute information can be extracted for each face image of the object, and / or body attribute information can be extracted for each body image of the object.

[0137] The aforementioned facial attributes include facial orientation, whether the face is obscured, hairstyle type, and facial score, while the aforementioned human body attributes may include human body orientation, whether the human body is obscured, clothing type, and clothing color.

[0138] After extracting the attribute information of each object screen of the object, it is possible to further filter out object screens whose attribute information matches the preset attribute filtering rules from the object screens of the object based on the attribute information of each object screen.

[0139] The preset attribute filtering rules can be determined based on needs and experience. For example, if the object image is a face image and the face attribute information is the occlusion ratio of the face, then the filtering rule can be to filter out face images whose occlusion ratio is less than a preset threshold.

[0140] When the object screen includes face screens and / or body screens, face screens whose face attribute information conforms to preset face attribute filtering rules can be selected from each face screen of the object; and / or, body screens whose body attribute information conforms to preset body attribute filtering rules can be selected from each body screen of the object.

[0141] Both the facial attribute filtering rules and the human body attribute filtering rules can be determined based on requirements and experience. In one implementation, the facial attribute filtering rules can filter out the facial images with the highest facial scores, while the human body attribute filtering rules can filter out the facial images with the highest human body scores.

[0142] S603, take each selected object screen and its attribute information as the feature information of the object to which the object screen belongs;

[0143] In the selected object frames, each selected object frame and its attribute information can be used as the feature information of the object to which that object frame belongs. Optionally, the selected object frames and their attribute information can be combined according to a pre-defined format to obtain the feature information of the selected objects.

[0144] In the above-described solution provided by the embodiments of the present invention, since the trajectory of an object can be generated using only a camera, the trajectory of the object is generated seamlessly. Furthermore, each camera can acquire an image of the object and extract the attribute information of each image to determine the characteristic information of the object, thereby providing a basis for the seamless generation of the object's trajectory.

[0145] Optionally, in one embodiment, the aforementioned management device is specifically used to input the trajectory within the field of view of each camera, as well as the feature information and identity information corresponding to each trajectory, into a pre-trained trajectory association model, so as to use the trajectory association model to determine the trajectories belonging to the same object from the trajectories within the field of view of each camera; wherein, the trajectory association model is trained using multiple sample trajectories, the feature information and identity information corresponding to each sample trajectory, and the calibration information of each sample trajectory, and the sample calibration information of each sample trajectory indicates the sample trajectory belonging to the same object as the sample trajectory.

[0146] Optionally, the management device is also used to, after performing trajectory fusion on each trajectory belonging to the object to obtain the trajectory of the object in the target scene, determine the identity information of the object for each object based on the identity information corresponding to the trajectory within the field of view of each camera belonging to the object, and use this as the identity information associated with the trajectory of the object in the target scene.

[0147] Optionally, the management device is specifically used to select the identity information with the highest proportion from the identity information corresponding to the trajectory within the field of view of each camera belonging to the object, and use it as the identity information of the object.

[0148] Since the management device in the trajectory generation system provided in this embodiment of the invention corresponds to the method provided from the perspective of the management device described above, the description is relatively simple. For relevant details, please refer to the description of the method provided from the perspective of the management device.

[0149] In the technical solution of this application, the acquisition, storage, use, processing, transmission, provision and disclosure of relevant data such as object information, feature information, trajectory and identity information are all carried out with the authorization of the object.

[0150] Corresponding to the methods provided above from the perspective of equipment management, such as Figure 7 As shown, this embodiment of the invention also provides a trajectory generation device, applied to a management device in a trajectory generation system. The trajectory generation system further includes: multiple cameras deployed in a target scene. The device includes:

[0151] The information receiving module 701 is used to receive object information generated by each camera for the captured object. The object information of each camera includes: the trajectory and feature information of the object within the field of view of the camera.

[0152] The information recognition module 702 is used to identify the identity information of the object to which the trajectory belongs based on the feature information corresponding to the trajectory for the trajectory of each object within the field of view of the camera, and use the feature information corresponding to the trajectory as the identity information corresponding to the trajectory; wherein the feature information corresponding to each trajectory is: the feature information of the object to which the trajectory belongs;

[0153] The trajectory determination module 703 is used to determine the trajectories belonging to the same object from the trajectories within the field of view of each camera based on each trajectory and the feature information and identity information corresponding to each trajectory.

[0154] The trajectory fusion module 704 is used to fuse the trajectories belonging to each object for each object, so as to obtain the trajectory of the object in the target scene.

[0155] Optionally, the trajectory determination module is specifically used to input the trajectories within the field of view of each camera, as well as the feature information and identity information corresponding to each trajectory, into a pre-trained trajectory association model, so as to use the trajectory association model to determine the trajectories belonging to the same object from the trajectories within the field of view of each camera; wherein, the trajectory association model is trained using multiple sample trajectories, the feature information and identity information corresponding to each sample trajectory, and the calibration information of each sample trajectory, and the sample calibration information of each sample trajectory indicates the sample trajectory belonging to the same object as the sample trajectory.

[0156] Optionally, the device further includes:

[0157] The identity determination module is used to determine the identity information of each object based on the identity information corresponding to the trajectory within the field of view of each camera belonging to the object after the trajectory fusion module performs trajectory fusion on each trajectory belonging to the object to obtain the trajectory of the object in the target scene. This identity information is used as the identity information associated with the trajectory of the object in the target scene.

[0158] Optionally, the identity determination module is specifically used to select the identity information with the highest proportion from the identity information corresponding to the trajectory within the field of view of each camera belonging to the object, and use it as the identity information of the object.

[0159] In the device provided by the embodiments of the present invention, since it can receive the trajectory and feature information generated by each camera for the captured object, and then determine the trajectories belonging to the same object from the trajectories within the field of view of each camera based on the trajectory and feature information of each object, and obtain the trajectory of each object in the target scene through trajectory fusion, the trajectory of the object can be generated by the camera alone, thus realizing the seamless generation of the object's trajectory.

[0160] This invention also provides an electronic device, such as... Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803, and a communication bus 804. The processor 801, communication interface 802, and memory 803 communicate with each other via the communication bus 804.

[0161] Memory 803 is used to store computer programs;

[0162] The processor 801, when executing the program stored in the memory 803, implements the method steps provided by the embodiments of the present invention from the perspective of a management device.

[0163] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0164] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0165] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0166] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0167] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of any of the above trajectory generation methods.

[0168] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the trajectory generation methods described above.

[0169] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0170] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0171] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device, equipment, and system embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0172] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A trajectory generation method characterized by, The application discloses a management device applied to a trajectory generation system, the trajectory generation system further comprising a plurality of cameras deployed in a target scene, and the method comprises the following steps: receiving object information generated for a photographed object sent by each camera, the object information of each camera comprising a trajectory and feature information of an object in the field of view of the camera, and the feature information comprising information representing generation information of the object; for the trajectory of an object in the field of view of each camera, identifying identity information of the object to which the trajectory belongs based on the feature information corresponding to the trajectory as identity information corresponding to the trajectory; wherein the feature information corresponding to each trajectory is feature information of the object to which the trajectory belongs; inputting the trajectories in the field of view of each camera and the feature information and identity information corresponding to each trajectory into a pre-trained trajectory association model to determine trajectories belonging to the same object from the trajectories in the field of view of each camera by using the trajectory association model; wherein the trajectory association model is trained by using a plurality of sample trajectories, feature information and identity information corresponding to each sample trajectory, and calibration information of each sample trajectory, and the sample calibration information of each sample trajectory indicates sample trajectories belonging to the same object as the sample trajectory; for each object, performing trajectory fusion on the trajectories belonging to the object to obtain a trajectory of the object in the target scene.

2. The method of claim 1, wherein, After the trajectories belonging to the object are fused to obtain the trajectory of the object in the target scene, the method further comprises the following steps: for each object, determining identity information of the object based on the identity information corresponding to the trajectories in the field of view of each camera belonging to the object as identity information associated with the trajectory of the object in the target scene.

3. The method of claim 2, wherein, The determination of the identity information of the object based on the identity information corresponding to the trajectories in the field of view of each camera belonging to the object comprises the following steps: selecting identity information with the largest proportion from the identity information corresponding to the trajectories in the field of view of each camera belonging to the object as the identity information of the object.

4. A trajectory generation system characterized by, The trajectory generation system comprises a management device and a plurality of cameras deployed in a target scene, wherein: a target camera is configured to generate object information of a photographed object, wherein the object information of the photographed object comprises a trajectory and feature information of the photographed object in the field of view of the target camera, and the feature information comprises information representing generation information of the object; and the target camera is any camera in the trajectory generation system. The management device is configured to, after receiving the object information sent by each camera in the trajectory generation system, identify, for each trajectory of an object in the field of view of each camera, identity information of the object to which the trajectory belongs as identity information corresponding to the trajectory based on feature information corresponding to the trajectory; input the trajectories in the field of view of each camera and the feature information and the identity information corresponding to each trajectory into a pre-trained trajectory association model, to determine, by using the trajectory association model, trajectories belonging to the same object from the trajectories in the fields of view of the cameras; and perform trajectory fusion on the trajectories belonging to each object to obtain a trajectory of the object in the target scene, where the feature information corresponding to each trajectory is feature information of the object to which the trajectory belongs, and the trajectory association model is trained by using a plurality of sample trajectories, feature information and identity information corresponding to each sample trajectory, and labeling information of each sample trajectory, and the sample labeling information of each sample trajectory indicates sample trajectories belonging to the same object as the sample trajectory.

5. The trajectory generation system of claim 4, wherein, The target camera generates the trajectories of the objects in the field of view of the camera in the following manner: record position information of the objects in the field of view according to a specified position acquisition period; generate, for each object in the field of view, a trajectory of the object in the field of view of the camera based on the position information of the object recorded at each position acquisition time.

6. The trajectory generation system of claim 5, wherein, The target camera is specifically configured to perform object detection on the collected image, and when an object is detected in the image, record position information of the detected object at each position acquisition time according to a specified position acquisition period.

7. The trajectory generation system of claim 6, wherein, The target camera is specifically configured to perform face detection, head-shoulder detection, and human body detection on the collected image, and when a face, a head-shoulder, and a human body are detected, associate the detected face, head-shoulder, and human body, and if there are associated faces, head-shoulders, and human bodies, take the associated face, head-shoulder, and human body as the detected object.

8. The trajectory generation system of any one of claims 5-7, wherein, The position information is a first coordinate of the object in a camera coordinate system. The target camera is specifically configured to generate, according to the order of the position acquisition times, a first coordinate linked list of the object based on the first coordinate of the object recorded at each position acquisition time, and convert each first coordinate in the first coordinate linked list into a second coordinate in a scene coordinate system of the target scene based on a mapping relationship between the camera coordinate system and the scene coordinate system to obtain a second coordinate linked list as a trajectory of the object in the field of view.

9. The trajectory generation system of claim 4, wherein, The target camera is specifically configured to generate the feature information of the object in the field of view in the following manner: At each acquisition moment, an object picture of an object in a field of view of the target camera is acquired; for each object in the field of view of the target camera, attribute information in each object picture of the object is extracted, and based on the attribute information in each object picture of the object, object pictures of the object in which the attribute information meets a preset attribute screening rule are screened out from the object pictures of the object; and each screened object picture and attribute information of the object picture are taken as feature information of the object to which the object picture belongs.

10. A trajectory generation device characterized by comprising: The management device is applied to a trajectory generation system, and the trajectory generation system further includes a plurality of cameras deployed in a target scene. An information receiving module is configured to receive object information generated for an object photographed by each camera, and the object information of each camera includes a trajectory and feature information of an object in a field of view of the camera, wherein the feature information includes information representing generation information of the object. An information identifying module is configured to identify identity information of an object to which a trajectory of an object in a field of view of each camera belongs as identity information corresponding to the trajectory based on feature information corresponding to the trajectory, and the feature information corresponding to each trajectory is feature information of the object to which the trajectory belongs. A trajectory determining module is configured to input the trajectories in the field of view of each camera and the feature information and identity information corresponding to each trajectory to a pre-trained trajectory association model, so as to determine trajectories belonging to the same object from the trajectories in the field of view of each camera by using the trajectory association model, wherein the trajectory association model is trained by using a plurality of sample trajectories, feature information and identity information corresponding to each sample trajectory, and labeling information of each sample trajectory, and the sample labeling information of each sample trajectory indicates sample trajectories belonging to the same object as the sample trajectory. A trajectory fusion module is configured to perform trajectory fusion on the trajectories belonging to each object to obtain a trajectory of the object in the target scene.

11. An electronic device, comprising: The device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. The memory is configured to store a computer program. The processor is configured to execute the program stored in the memory to implement the method steps in any one of claims 1-3.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method steps in any one of claims 1-3.

Images