Target human body cross-lens re-identification method and system in closed environment and electronic device

Abstract

The application discloses a target human body cross-lens re-identification method and system in a closed environment and electronic equipment, and belongs to the technical field of cross-lens re-identification.The application takes an image containing a target object in second video data stream as a template image, takes a current frame of the first video data stream as a search image, inputs the template image and the search image into a twin model, calculates a similarity score value of feature vectors of the two images, and obtains a trajectory segment of the human body according to the score value to mine time information in the video.Then, based on pixel coordinates and label information of each human body in each frame image, spatial information in the video is mined to obtain position information of each human body, and then it is judged whether the spatial distance of the human body with the same label in the current frame and the previous frame image is less than a threshold value, so that the lost target can be quickly found, and the performance of the target tracking method is improved.The technical problem that the target cannot be re-identified after the target is lost in the cross-border process in a closed complex scene is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cross-lens re-identification, and more particularly, to a target human cross-lens re-identification method, system and electronic device in a closed environment. BACKGROUND

[0002] Cross-lens re-identification refers to a method of searching and tracking a pedestrian appearing in two non-overlapping cameras at different time nodes. For example, camera A and camera B are two cameras that have no intersection. A motion video data or a single frame image of a pedestrian is captured in camera A, and then the pedestrian captured in camera A is searched and identified in the image captured by camera B. Whether the pedestrian captured in camera A appears in the data captured by camera B is observed. Camera A and camera B are independent of each other. This searching and identifying method is called cross-lens re-identification.

[0003] For example, the application CN112001251A discloses a pedestrian re-identification method and system based on human parsing and clothing color combination, which comprises: acquiring a target pedestrian image to be identified; inputting the target pedestrian image to be identified into a pre-constructed human parsing model to output the body part categories of the target pedestrian to be identified; extracting color features of each category of body part from the target pedestrian image to be identified; and identifying the target pedestrian from the candidate images according to the color features of each category of body part.

[0004] However, in a closed complex scene, the background of pedestrians is complex and variable. When a computer tracks a pedestrian, the pedestrian is easily lost due to occlusion and other problems, and cannot be re-identified after the target is lost. There is no corresponding solution to the problem that the target cannot be re-identified after being lost in the cross-border process in a closed complex scene. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides a target human cross-lens re-identification method, system and electronic device in a closed environment.

[0006] According to one aspect of the present application, a target human cross-lens re-identification method in a closed environment is provided, comprising:

[0007] acquiring a target image and a first video data stream to be cross-lens re-identified; wherein the first video data stream is acquired by a first image acquisition device in a closed environment, the target image is any frame image containing a target object in a second video data stream, the second video data stream is acquired by a second image acquisition device in a closed environment, and the first image acquisition device and the second image acquisition device are independent of each other;

[0008] input the template image and the search image into a pre-trained twin model, and calculate a similarity score value between the feature vector of the template image and the feature vector of the search image; wherein the twin model has two branches with the same architecture and weight set, one of which is used to learn the feature representation of the target, and the other is used to search the region; and the twin model is trained by using a random patch adding strategy; the twin model calculates the similarity score value between the feature vector of the template image and the feature vector of the search image by using the following function:

[0009]

[0010] In the formula, f θ (z, x) represents a similarity response function of the feature vector of the template image z and the feature vector of the search image x in the twin model containing the parameter θ; (z, x) represents a similarity response function of the feature vector of the template image z and the feature vector of the search image x in the twin model containing the parameter θ; (z, x) represents a similarity response function of the feature vector of the template image z and the feature vector of the search image x in the twin model containing the parameter θ;

[0011] The similarity score value is compared with a preset similarity threshold value to determine whether the target human body with the same identity as the target object exists in the current frame image; when the target human body with the same identity as the target object exists in the current frame image, the label of the target human body in the current frame image is determined according to the label of the target object, and the pixel coordinates of the target human body in the current frame image are recorded; when other human bodies in addition to the target human body are recognized in the current frame image, the pixel coordinates and label information of the other human bodies are determined.

[0012] Based on the pixel coordinates and label information of each human body in each frame image, it is determined whether the spatial distance between the human bodies with the same label in the current frame image and the previous frame image is less than a preset distance threshold value; when the spatial distance between the human bodies with the same label in the current frame image and the previous frame image is less than the preset distance threshold value, it is determined that the target human body with the same identity as the target object exists in the first video data stream.

[0013] Optionally, the target image containing the target object is obtained, comprising:

[0014] The second video data stream is identified to determine each frame image containing the target object in the second video data stream.

[0015] Based on each frame image containing the target object, a template image library is constructed.

[0016] When target human body cross-camera re-identification is needed, a frame image is randomly extracted from the constructed template image library as a target image.

[0017] Optionally, the method further includes:

[0018] Before training the twin model, black patches of preset areas are randomly placed in the two sample images to mask the most distinctive parts of the two sample images. The first sample image is the original image collected, and the second sample image is an image containing the human body area obtained by cropping the first sample image.

[0019] During the training process, two sample images including black patches are input into each branch of the twin model, and the feature calculation results output by each branch are all linked. Then, they are input into the linear fully connected layer of the twin model and the training results are output.

[0020] Optionally, the method also includes: when the spatial distance between the human body with the same label in the current frame image and the previous frame image is not less than a preset distance threshold, re-determining the label of the corresponding human body in the current frame image, and deducting the image of the corresponding human body and storing it in the template image library.

[0021] Optionally, after determining that a target human body with the same identity as the target object exists in the video data stream, the method further includes: deducting an image of the target human body and incorporating it into a template image library.

[0022] According to another aspect of the present invention, a system for cross-shot human target re-identification in a closed environment is provided, comprising:

[0023] an acquisition module, configured to acquire a target image and a first video data stream to be re-identified across shots; wherein the first video data stream is acquired by a first image acquisition device in a closed environment, the target image is any frame of an image containing a target object in a second video data stream, and the second video data stream is acquired by a second image acquisition device in a closed environment, the first image acquisition device and the second image acquisition device being independent of each other;

[0024] A similarity calculation module is configured to use the target image as a template image and the current frame image of the first video data stream as a search image, input the template image and the search image into a pre-trained twin model, and calculate the similarity score between the feature vector of the template image and the feature vector of the search image; wherein the twin model has two branches with the same architecture and weight set, one branch is used to learn the feature representation of the target, and the other branch is used to search the area; and the twin model is trained using a random patch addition strategy; the twin model uses the following function to calculate the similarity score between the feature vector of the template image and the feature vector of the search image:

[0025]

[0026] Where, fθ (z, x) represents a similarity response function of the feature vector of the template image z and the feature vector of the search image x in a twin model containing parameters θ; represents the feature vector of the template image z in a twin model containing parameters θ, represents the feature vector of the search image x in a twin model containing parameters θ, b·l represents a bias term at each position b;

[0027] The first judging module is configured to compare the similarity score value with a preset similarity threshold value, and determine whether the target human body with the same identity as the target object exists in the current frame image; when the target human body with the same identity as the target object exists in the current frame image, the label of the target human body in the current frame image is determined according to the label of the target object, and the pixel coordinates of the target human body in the current frame image are recorded; when it is identified that there is a human body other than the target human body in the current frame image, the pixel coordinates and label information of the other human body are determined.

[0028] The second judging module is configured to determine, based on the pixel coordinates and label information of each human body in each frame image, whether the spatial distance between the human bodies with the same label in the current frame image and the previous frame image is less than a preset distance threshold value; when the spatial distance between the human bodies with the same label in the current frame image and the previous frame image is less than the preset distance threshold value, it is determined that the target human body with the same identity as the target object exists in the first video data stream.

[0029] Optionally, the acquisition module is specifically configured to:

[0030] identify the second video data stream to determine the frames of images containing the target object in the second video data stream;

[0031] construct a template image library based on the frames of images containing the target object;

[0032] When target human body cross-camera re-identification is needed, a frame of image is randomly extracted from the constructed template image library as a target image.

[0033] Optionally, the system further includes a training module configured to:

[0034] Before training the twin model, a black patch of a preset area is randomly placed in two sample images to cover the most distinctive part of the two sample images; wherein the first sample image is an original image collected, and the second sample image is an image containing a human body region obtained by cropping the first sample image;

[0035] In the training process, two sample images including black patches are input into each branch of the twin model, the feature calculation results output by each branch are all linked, and then input into a linear fully connected layer of the twin model, and the training result is output.

[0036] According to still another aspect of the present application, there is provided a computer readable storage medium storing a computer program for executing the method according to any one of the preceding aspects of the present application.

[0037] According to still another aspect of the present application, there is provided an electronic device comprising a processor, a memory for storing instructions executable by the processor, and the processor configured to read the executable instructions from the memory and execute the executable instructions to implement the method according to any one of the preceding aspects of the present application.

[0038] The present application improves the accuracy of the target re-identification method by taking any frame image containing a target object in the second video data stream as a template image, taking the current frame image of the first video data stream as a search image, inputting the template image and the search image into a pre-trained twin model, calculating the similarity score value between the feature vector of the template image and the feature vector of the search image, and mining the time information in the video to obtain the trajectory segment of the human body. Further, based on the pixel coordinates and label information of each human body in each frame image, the spatial information in the video is mined to obtain the position information of each human body, and according to the position information, it is judged whether the spatial distance between the human bodies with the same label in the current frame image and the previous frame image is less than a preset distance threshold, so as to quickly find the lost target and improve the performance of the target tracking method. The technical problem of being unable to re-identify the target after the target is lost in the cross-border head process in a closed complex scene is solved. BRIEF DESCRIPTION OF DRAWINGS

[0039] The exemplary embodiments of the present application can be more completely understood in reference to the following drawings:

[0040] Figure 1 is a flowchart of a target human body cross-lens re-identification method in a closed environment provided by an exemplary embodiment of the present application;

[0041] Figure 2 is a structural diagram of a target human body cross-lens re-identification system in a closed environment provided by an embodiment of the present application;

[0042] Figure 3 is a structure of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0043] Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and thus are not to limit the whole embodiments of the present application, and it should be appreciated that the present application covers all the technical solutions which can be obtained by those of ordinary skill in the art based on the embodiments of the present application without making creative efforts.

[0044] It should be noted that the relative arrangement of the components and steps, numerical expressions, and numerical values set forth in the embodiments are not limitations on the scope of the present application unless otherwise specifically indicated.

[0045] Those skilled in the art can understand that the terms "first", "second" and the like in the embodiments of the present application are only used to distinguish different steps, devices or modules and the like, and do not represent any specific technical meaning, nor indicate their logical order.

[0046] It should also be understood that in the embodiments of the present application, "a plurality of" can mean two or more, and "at least one" can mean one, two or more.

[0047] It should also be understood that for any component, data or structure mentioned in the embodiments of the present application, it can be understood as one or more in general, without explicit limitation or in the context of the opposite indication.

[0048] In addition, the term "and / or" in the present application is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. In addition, the character " / " in the present application generally represents an "or" relationship between the front and rear associated objects.

[0049] It should also be understood that the description of the embodiments of the present application emphasizes the differences between the embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0050] At the same time, it should be understood that in order to facilitate the description, the size of each part shown in the drawings is not drawn according to the actual proportional relationship.

[0051] The following description of at least one example embodiment is merely illustrative in nature and is in no way limiting to the application or its use.

[0052] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail herein. However, where appropriate, the techniques, methods, and devices should be considered as part of the description of the present application.

[0053] It should be noted that similar reference numbers and letters in the following drawings represent similar items, and thus, once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0054] Embodiments of the application can be applied to electronic devices such as communication terminals, computer systems, servers, etc., which can operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known communication terminals, computing systems, environments, and / or configurations that can be suitable for use with electronic devices such as communication terminals, computer systems, servers, etc., include, but are not limited to, personal computers, server computers, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed cloud computing technology environments that include any of the above systems, and the like.

[0055] Electronic devices such as communication terminals, computer systems, servers, etc., can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, and the like, which perform particular tasks or implement particular abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules can be located in local or remote computer system storage media including storage devices.

[0056] The present application provides a target human body cross-lens re-identification method in a closed environment, a system and an electronic device. Figure 1 is a flowchart of a target human body cross-lens re-identification method in a closed environment provided by an exemplary embodiment of the present application. As shown in Figure 1 The target human body cross-lens re-identification method in a closed environment includes:

[0057] Step S1: obtaining a target image and first video data stream to be cross-lens re-identified; wherein the first video data stream is collected by a first image collection device in a closed environment, the target image is any frame image containing a target object in a second video data stream, the second video data stream is collected by a second image collection device in a closed environment, and the first image collection device and the second image collection device are independent of each other;

[0058] Optionally, the target image containing the target object is obtained by: identifying the second video data stream to determine each frame image containing the target object in the second video data stream; constructing a template image library based on each frame image containing the target object; and when target human body cross-lens re-identification is needed, randomly extracting a frame image from the constructed template image library as the target image.

[0059] In the embodiment of the present application, first, a target image containing a target object and a first video data stream to be cross-lens re-identified are required. Specifically, a first video data stream to be cross-lens re-identified is collected through a first image collection device, a second video data stream is collected through a second image collection device, then the second video data stream is identified to determine each frame image containing the target object in the second video data stream, and based on each frame image containing the target object, a template image library is constructed, and when target human body cross-lens re-identification is required, a frame image is randomly extracted from the constructed template image library as a target image.

[0060] Step S2: the current frame image of the first video data stream is taken as a search image, the template image and the search image are input into the pre-trained twin model, and the similarity score value between the feature vector of the template image and the feature vector of the search image is calculated; wherein the twin model has two branches with the same architecture and weight set, one of which is used to learn the feature representation of the target, and the other is used to search the region; and the twin model is trained by using a random patch adding strategy; the twin model calculates the similarity score value between the feature vector of the template image and the feature vector of the search image by using the following function:

[0061]

[0062] In the formula, f θ (Z,x) represents the similarity response function of the feature vector of the template image z and the feature vector of the search image x in the twin model containing the parameter θ; represents the feature vector of the template image z in the twin model containing the parameter θ, represents the feature vector of the search image x in the twin model containing the parameter θ, and b·l represents the bias term at each position b;

[0063] Optionally, the method further comprises: before training the twin model, randomly placing a black patch of a preset region in two sample images, and masking the most distinctive part in the two sample images; wherein the first sample image is an original image collected, and the second sample image is an image containing a human body region obtained by cropping the first sample image; in the training process, the two sample images including the black patch are input into each branch of the twin model, the feature calculation results output by each branch are all linked, and then input into the linear fully connected layer of the twin model, and the training result is output.

[0064] In the embodiment of the present application, a tracking similar graph is cited from a deep model with a twin network structure, one branch is used to learn the feature representation of the target, and the other branch is used to search the region. There are two branches in the network that share the same architecture and the same weight set, and during the training process, after the video stream is input into the twin network, a black patch of a preset region is randomly placed in the image region, which is used to ensure that the most distinctive part of an object is not always visible to the network, so as to force it to also pay attention to other relevant parts of the object. In this way, the limitation of the existing semi-supervised method that only pays attention to the most distinctive part of the object can be overcome, and the generalization ability of the model can be improved.

[0065] During the training process, each training branch takes the two images containing the patch as input, and then performs a series of convolution operations. The feature calculation results output by the branches are all linked and input into the top network composed of linear fully connected layers. The training process repeats the above process, and finally an optimal convolutional neural network containing parameters θ is obtained, so as to obtain a trained twin model.

[0066] Further, the target image is taken as a template image z, and the current video stream is taken as a search image x. The two images are input into the convolutional neural network with parameters θ for processing, to generate two interrelated feature maps:

[0067]

[0068] Wherein, f θ (z,x) represents the similarity response function of the feature vector of the template image z and the feature vector of the search image x in the convolutional neural network with parameters θ; represents the feature vector of the template image z in the convolutional neural network with parameters θ, represents the feature vector of the search image x in the convolutional neural network with parameters θ, and b·l represents the bias term of the value at each position b. Formula (1) is equivalent to an exhaustive search of the region z of x on the image, and the goal is to match the maximum value in the response mapping f to the target position. In order to achieve this goal, the network obtains the parameters θ of the convolutional neural network by minimizing the logical loss of the training set through random image pairs (z, x) and corresponding real position labels y:

[0069]

[0070] When designing the tracker, this simple matching function has two inherent limitations, which are: the feature extractor used in the tracker has an inherent constraint of strict translation invariance f(z,x[Δτ j ])=f(z,x)[Δτ jwhere f(z,x) represents the similarity response function of the feature vector of the template image z and the feature vector of the search image x, and the translation displacement sub-window operator ensures effective training and inference; the intrinsic restriction of structural symmetry f(z,x') = f(x',z) is ensured.

[0071] where z represents the template image, and the algorithm uses the first frame of ground truth; x represents the search region, which indicates the candidate search region in the subsequent frame to be tracked; represents the feature mapping operation, which maps the original image to a specific feature space through the convolution layer and the pooling layer in the neural network; 6x6x128 represents the feature obtained after z is processed by the program ; it is a 128-channel feature with a size of 6x6; similarly, 22x22x128 is the subsequent feature size obtained after x is processed by the program ; the feature in 22x22x128 is convolved by the 6x6x128 convolution kernel, and then a 17x17 score map is obtained, i.e., the similarity value of each image pair in the search region. The neural network itself compares the difference between the target image search region and the template image, and thus obtains the score value of the search region.

[0072] Step S3: comparing the similarity score value with a preset similarity threshold value to determine whether there is a target human body with the same identity as the target object in the current frame image; when there is a target human body with the same identity as the target object in the current frame image, determining the label of the target human body in the current frame image according to the label of the target object, and recording the pixel coordinates of the target human body in the current frame image; when other human bodies in addition to the target human body are recognized in the current frame image, determining the pixel coordinates and label information of the other human bodies;

[0073] Optionally, after it is determined that there is a target human body with the same identity as the target object in the video data stream, the method further includes: extracting the image of the target human body and storing it in the template image library.

[0074] In the embodiment of the application, by presetting a similarity score threshold value, when the score value is less than the threshold value, it is determined that there is no target with the same identity as the template image in the current search image, and the target of interest in the current video stream is extracted (using a common target detection method) and stored in the template image library; if the score is not less than the threshold value, it is preliminarily determined that there is a target with the same identity as the template image in the current search image.

[0075] Step S4: based on the pixel coordinates and label information of each human body in each frame image, it is judged whether the spatial distance of the human bodies with the same label in the current frame image and the previous frame image is less than a preset distance threshold; when the spatial distance of the human bodies with the same label in the current frame image and the previous frame image is less than the preset distance threshold, it is determined that there is a target human body with the same identity as the target object in the first video data stream.

[0076] Optionally, the method further comprises: when the spatial distance of the human bodies with the same label in the current frame image and the previous frame image is not less than the preset distance threshold, re-determining the label of the corresponding human body in the current frame image, and deducting the image of the corresponding human body and storing it in the template image library.

[0077] In the embodiment of the application, although the random patch method is adopted when training the model to effectively reduce the number of times of personnel identification and error identification caused by personnel partial occlusion, when the personnel are seriously occluded, the above method still cannot accurately identify the identity information of the personnel. Therefore, a subsequent correction method based on spatial information is added, and the specific process is as follows:

[0078] 1) Save the pixel coordinates and label information of each target correctly identified in the current search image x, which is used for correction with the next frame.

[0079] 2) Judge whether the spatial distance of the same label targets in the current frame and the previous frame is less than the threshold value. The distance is calculated by the Euclidean distance of the center point coordinates of the two detection boxes.

[0080] 3) If the distance is greater than or equal to the threshold value, it is preliminarily determined that the same target in the previous frame and the current frame is not the same person, the target in the current frame is reset as a new label, and the target image is deducted and stored in the template image library; if the distance is less than the threshold value, the target label information is not corrected, it is considered that the preliminary determination by the twin network is correct, the output result is output, and the current target is deducted and stored in the template image library.

[0081] 4) The expanded template image library has template images of different labels in different time sequences, and the f θ (z,x) of the same class label in different time sequences is calculated, low-score template images in the same class label are dynamically removed in real time, and the accuracy of re-identification is improved.

[0082] The application adopts a twin network as a basic network for extracting position information features, and in view of the problem of "person occlusion", a random mask block is connected to the target object of the training set, so that some areas with distinguishing features of the target person will not affect the training of the overall features, and the accuracy of pedestrian tracking is improved. The application introduces a stable twin network tracking method based on semi-supervised learning, adopts a position-enhanced twin network model and a random occlusion attack strategy to solve the problem of occluded pedestrian samples; the application explores a progressive pedestrian re-identification network updating method based on reinforcement learning, adopts a network cooperative training strategy based on reinforcement learning and a video sequence inter-frame information mining model to cope with the insufficient information mining between video streams.

[0083] The application improves the problem of complex and variable pedestrian background, based on the existing cross-border head re-identification technology, extracts the trajectory segment of the pedestrian in the video, expands the pseudo-label video segment dataset, and improves the accuracy of the pedestrian re-identification method; the pedestrian re-identification technology is used to extract the spatial information of the pedestrian in the video to obtain the position segment of the pedestrian, due to the characteristics of the pedestrian re-identification, the lost target can be quickly found and recorded as a difficult sample, so that the performance of the target tracking method is improved. Relying on this space-time information joint system optimization method, the target tracking network and the pedestrian re-identification network can promote each other to form a virtuous cycle, and the overall performance of the network is further improved.

[0084] Therefore, the application can be used as a common visual matching algorithm to provide an auxiliary scheme for other computer vision tasks. For example, when using face recognition technology for identity recognition, there may be a problem that the face image of the pedestrian captured in the cross-time and space video monitoring system is not high-definition, at which time the identity recognition cannot be correctly performed, and the cross-lens re-identification technology can be used as an auxiliary scheme to improve the accuracy of the pedestrian identity recognition system in a complex interference scene according to the appearance, clothing and other information of the obtained pedestrian.

[0085] The application of the application can fully identify the trajectory of the target human body movement and stay, track and analyze the behavior of the target human body, for example, when the computer tracks the pedestrian, the person will be lost due to the problem of occlusion, and the cross-lens re-identification method can reposition the target human body, so that the computer can continuously perform the tracking task.

[0086] The algorithm research idea of the application can inspire the research and development of other visual retrieval matching tasks, such as video image retrieval, image classification and face matching, and other hot technologies, so as to promote the development and progress of the related research of computer vision tasks.

[0087] In summary, the present application is based on the existing cross-border head recognition technology, takes any frame image containing a target object in the second video data stream as a template image, takes the current frame image of the first video data stream as a search image, inputs the template image and the search image into the pre-trained twin model, calculates the similarity score value between the feature vector of the template image and the feature vector of the search image, and mines the time information in the video to obtain the trajectory segment of the human body, so that the accuracy of the target re-identification method is improved. Further, based on the pixel coordinates and label information of each human body in each frame image, the spatial information in the video is mined to obtain the position information of each human body, and whether the spatial distance of the human bodies with the same label in the current frame image and the last frame image is less than the preset distance threshold is judged according to the position information, so that the lost target is quickly found, and the performance of the target tracking method is improved. The technical problem that the target cannot be re-identified after the target is lost in the cross-border head process in a closed complex scene is solved.

[0088] Exemplary system

[0089] Figure 2 is a structural schematic diagram of a cross-lens re-identification system for a target human body in a closed environment provided by an exemplary embodiment of the present application. As shown in Figure 2 , the system comprises:

[0090] The acquisition module 210 is configured to acquire a target image and a first video data stream to be cross-lens re-identified. The first video data stream is acquired by a first image acquisition device in a closed environment, and the target image is any frame image containing a target object in a second video data stream, which is acquired by a second image acquisition device in a closed environment. The first image acquisition device and the second image acquisition device are independent of each other.

[0091] The similarity calculation module 220 is configured to take the target image as a template image, take the current frame image of the first video data stream as a search image, input the template image and the search image into a pre-trained twin model, and calculate the similarity score value between the feature vector of the template image and the feature vector of the search image. The twin model has two branches with the same architecture and weight set, one of which is used to learn the feature representation of the target, and the other of which is used to search the region. The twin model is trained by using a random patch adding strategy. The twin model calculates the similarity score value between the feature vector of the template image and the feature vector of the search image by using the following function:

[0092]

[0093] In the formula, f θ (z,x) represents the similarity response function of the feature vector of the template image z and the feature vector of the search image x in the twin model containing the parameter θ. represents a feature vector of a template image z in a twin model containing parameters θ, represents a feature vector of a search image x in a twin model containing parameters θ, b·l represents a bias term at each position b;

[0094] The first judging module 230 is configured to compare the similarity score value with a preset similarity threshold value, and determine whether there is a target human body with the same identity as the target object in the current frame image; when there is a target human body with the same identity as the target object in the current frame image, the label of the target human body in the current frame image is determined according to the label of the target object, and the pixel coordinates of the target human body in the current frame image are recorded; when it is identified that there is a human body other than the target human body in the current frame image, the pixel coordinates and label information of the other human body are determined.

[0095] The second judging module 240 is configured to determine, based on the pixel coordinates and label information of each human body in each frame image, whether the spatial distance between the human bodies with the same label in the current frame image and the previous frame image is less than a preset distance threshold value; when the spatial distance between the human bodies with the same label in the current frame image and the previous frame image is less than the preset distance threshold value, it is determined that there is a target human body with the same identity as the target object in the first video data stream.

[0096] Optionally, the acquisition module 210 is specifically configured to:

[0097] identify the second video data stream to determine each frame image containing the target object in the second video data stream;

[0098] construct a template image library based on the each frame image containing the target object;

[0099] When target human body cross-camera re-identification is needed, a frame image is randomly extracted from the constructed template image library as a target image.

[0100] Optionally, the system further includes a training module configured to:

[0101] Before training the twin model, a black patch of a preset area is randomly placed in two sample images, and the most distinctive part of the two sample images is covered; wherein the first sample image is an original image collected, and the second sample image is an image containing a human body region obtained by cropping the first sample image;

[0102] In the training process, the two sample images including the black patch are input into each branch of the twin model, the feature calculation results output by each branch are all linked, and then input into the linear fully connected layer of the twin model to output the training result.

[0103] The closed-environment target human cross-lens re-identification device of the embodiment of the present application corresponds to the closed-environment target human cross-lens re-identification method of another embodiment of the present application, which will not be described here again.

[0104] Exemplary electronic device

[0105] Figure 3 is a structure of an electronic device provided by an exemplary embodiment of the present application. As shown in Figure 3 , the electronic device 30 includes one or more processors 31 and a memory 32.

[0106] The processor 31 can be a central processing unit (CPU) or other form of processing unit that has data processing capability and / or instruction execution capability, and can control other components in the electronic device to perform desired functions.

[0107] The memory 32 can include one or more computer program products that can include various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM), cache memory, and / or the like. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program instructions can be stored on the computer readable storage media, and the processor 31 can execute the program instructions to implement the methods of the software programs of the various embodiments of the present application described above and / or other desired functions. In one example, the electronic device can further include an input device 33 and an output device 34, which are interconnected through a bus system and / or other form of connection mechanism (not shown).

[0108] In addition, the input device 33 can also include, for example, a keyboard, a mouse, and / or the like.

[0109] The output device 34 can output various information to the outside. The output device 34 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and / or the like.

[0110] Of course, in order to simplify, Figure 3 only some of the components of the electronic device related to the present application are shown in the above description, and components such as buses, input / output interfaces, and / or the like are omitted. In addition, the electronic device can also include any other appropriate components according to specific application cases.

[0111] Exemplary computer program product and computer readable storage medium

[0112] In addition to the methods and devices described above, embodiments of the present application can also be a computer program product which includes computer program instructions workable by a processor to cause the processor to carry out the steps of the methods according to various embodiments of the present application described in the above "Exemplary Methods" section of this specification when the computer program instructions are run by the processor.

[0113] The computer program instructions can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, etc., and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server.

[0114] In addition, embodiments of the present application can also be a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to carry out the steps of the methods according to various embodiments of the present application described in the above "Exemplary Methods" section of this specification.

[0115] The computer readable storage medium can be any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0116] The above describes the basic principles of the present application in conjunction with specific embodiments, but it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are only examples and are not limiting, and these advantages, benefits, effects, etc. cannot be considered as necessary for each embodiment of the present application. In addition, the above disclosed specific details are only for the purpose of example and understanding, and are not limiting, and the above details do not limit the present application to the above specific details.

[0117] The various embodiments described in this specification are presented for the purpose of illustration and description. Each of the embodiments highlights a different aspect of the other embodiments. The embodiments are not mutually exclusive, and portions of one or more embodiments can be combined with portions of one or more other embodiments. For system embodiments, the description is relatively brief because the system embodiments are generally similar to the method embodiments. The relevant portions of the method embodiments are described in the description of the method embodiments.

[0118] The block diagrams of the devices, systems, apparatuses, systems referred to in this specification are only meant to be illustrative examples and are not intended to require or imply that the devices, systems, apparatuses, systems be connected, arranged, configured in the manner as depicted in the block diagrams. These devices, systems, apparatuses, systems can be connected, arranged, configured in any manner as will be appreciated by those skilled in the art. Words such as "include," "contain," "have," etc. are open-ended words, meaning "including but not limited to," and are to be interpreted in the same manner as "comprising." The word "or" as used in this specification is intended to mean "and / or," unless otherwise indicated. The word "such as" is intended to mean "such as but not limited to," and is to be interpreted in the same manner as "comprising."

[0119] The methods and systems of the present application can be implemented in a number of ways. For example, the methods and systems of the present application can be implemented via software, hardware, firmware, or any combination of software, hardware, and firmware. The above described order of steps for the methods is merely illustrative, and the steps of the methods of the present application are not limited to the order described above unless otherwise specifically stated. Furthermore, in some embodiments, the present application can also be implemented as a program recorded on a recording medium, which includes machine readable instructions for implementing the methods according to the present application. Thus, the present application also covers recording media storing programs for executing the methods according to the present application.

[0120] It is also important to note that the systems, apparatuses, and methods of the present application can be embodied in a number of different ways. For example, the systems, apparatuses, and methods of the present application can be implemented or realized in hardware, software, firmware, or any combination thereof. The systems, apparatuses, and methods of the present application can be implemented or realized in a centralized fashion in one computer system or in a distributed fashion where different elements are spread across several computer systems. Any kind of computer system or other apparatus adapted for carrying out the methods of the application, as described herein, is suited. A typical combination of hardware and software could be a general purpose computer system with a computer program that, when being loaded and executed, controls the computer system such that it carries out the methods of the present application.

[0121] The above description is presented for the purpose of illustration and description. It is not intended to be exhaustive or to limit the embodiments of the present application to the precise forms disclosed. Although the various example aspects and embodiments have been discussed, it should be understood that certain modifications, changes, substitutions, additions and rearrangements by those skilled in the art can be made without departing from the scope of the application, as defined in the following claims.

Claims

1. A method for cross-camera re-identification of a target human body in a closed environment, characterized in that, The method comprises: acquiring a target image and a first video data stream to be cross-lens re-identified; wherein the first video data stream is acquired by a first image acquisition device in a closed environment, the target image is any frame image containing a target object in a second video data stream, the second video data stream is acquired by a second image acquisition device in the closed environment, and the first image acquisition device and the second image acquisition device are independent of each other; inputting the target image as a template image and a current frame image of the first video data stream as a search image into a pre-trained twin model, calculating a similarity score value between a feature vector of the template image and a feature vector of the search image; wherein the twin model has two branches with the same architecture and weight set, one of which is used to learn the feature representation of the target, and the other is used to search the region; and the twin model is trained by using a random patch adding strategy; the twin model calculates the similarity score value between the feature vector of the template image and the feature vector of the search image by using the following function: where f θ (z, x) denotes the similarity response function between the feature vector of the template image z and the feature vector of the search image x in a twin model containing parameters θ; denotes the feature vector of the template image z in a twin model containing parameters θ, denotes the feature vector of the search image x in a twin model containing parameters θ, b · l denotes the bias term at each position b; comparing the similarity score value with a preset similarity threshold value to determine whether there is a target human body with the same identity as the target object in the current frame image; when there is a target human body with the same identity as the target object in the current frame image, determining the label of the target human body in the current frame image according to the label of the target object, and recording the pixel coordinates of the target human body in the current frame image; when other human bodies in addition to the target human body are identified in the current frame image, determining the pixel coordinates and label information of the other human bodies; judging whether the spatial distance between the current frame image and the human body with the same label in the previous frame image is less than a preset distance threshold value based on the pixel coordinates and label information of each human body in each frame image; when the spatial distance between the current frame image and the human body with the same label in the previous frame image is less than the preset distance threshold value, it is determined that there is a target human body with the same identity as the target object in the first video data stream; and the method further comprises: before training the twin model, randomly placing black patches of a preset area in two sample images to cover the most distinctive parts in the two sample images; wherein the first sample image is an original image acquired, and the second sample image is an image containing a human body region obtained by cropping the first sample image; in the training process, inputting the two sample images including the black patches into each branch of the twin model, linking all the feature calculation results output by each branch, and then inputting the linear fully connected layer of the twin model to output the training result.

2. The method of claim 1, wherein, The method comprises: identifying the second video data stream to determine each frame image containing the target object in the second video data stream; based on each frame image containing the target object, a template image library is constructed; when target human body cross-lens re-identification is needed, a frame image is randomly extracted from the constructed template image library as a target image.

3. The method of claim 2, wherein, Further comprising: When the spatial distance of the human bodies with the same label in the current frame image and the previous frame image is not less than the preset distance threshold, the label of the corresponding human body in the current frame image is re-determined, and the image of the corresponding human body is deducted and stored in the template image library.

4. The method of claim 2, wherein, After determining that the target human body with the same identity as the target object exists in the video data stream, the image of the target human body is further deducted and stored in the template image library. 5.A cross-camera re-identification system for a target human body in a closed environment, characterized in that, The method comprises the steps of: The acquisition module is configured to acquire a target image and a first video data stream to be cross-shot re-identified, wherein the first video data stream is acquired by a first image acquisition device in a closed environment, and the target image is any frame image containing a target object in a second video data stream, which is acquired by a second image acquisition device in the closed environment, and the first image acquisition device and the second image acquisition device are independent of each other. The similarity calculation module is configured to take the target image as a template image, take a current frame image of the first video data stream as a search image, input the template image and the search image into a pre-trained twin model, and calculate a similarity score value between a feature vector of the template image and a feature vector of the search image, wherein the twin model has two branches with the same architecture and weight set, one of which is used to learn the feature representation of the target, and the other is used to search the region, and the twin model is trained by using a random patch adding strategy, and the twin model calculates the similarity score value between the feature vector of the template image and the feature vector of the search image by using the following function: where f θ (z, x) denotes the similarity response function between the feature vector of the template image z and the feature vector of the search image x in a twin model with parameters θ; denotes the feature vector of the template image z in a twin model with parameters θ, denotes the feature vector of the search image x in a twin model with parameters θ, b · l denotes the bias term at each position b; The first judgment module is configured to compare the similarity score value with a preset similarity threshold, and determine whether the target human body with the same identity as the target object exists in the current frame image, and when the target human body with the same identity as the target object exists in the current frame image, determine the label of the target human body in the current frame image according to the label of the target object, and record the pixel coordinates of the target human body in the current frame image, and when other human bodies in addition to the target human body are identified in the current frame image, determine the pixel coordinates and label information of the other human bodies. The second judgment module is configured to determine whether the spatial distance of the human bodies with the same label in the current frame image and the previous frame image is less than a preset distance threshold based on the pixel coordinates and the label information of each human body in each frame image, and when the spatial distance of the human bodies with the same label in the current frame image and the previous frame image is less than the preset distance threshold, determine that the target human body with the same identity as the target object exists in the first video data stream. The system further comprises a training module configured to: Before training the twin model, a black patch of a preset area is randomly placed in two sample images, and the most distinctive part in the two sample images is covered, wherein the first sample image is an original image acquired, and the second sample image is an image containing a human body region obtained by cropping the first sample image. In the training process, the two sample images including the black patch are input into each branch of the twin model, the feature calculation results output by each branch are all linked, and then input into a linear fully connected layer of the twin model to output the training result.

6. The system of claim 5, wherein, The acquisition module is specifically configured to: identifying the second video data stream to determine each frame image containing the target object in the second video data stream; constructing a template image library based on each frame image containing the target object; when target human body cross-camera re-identification is needed, randomly extracting a frame image from the constructed template image library as a target image. 7.A computer readable storage medium, the storage medium storing a computer program, the computer program being used to execute the method of any one of claims 1-4.

8. An electronic device, comprising: The electronic device comprises: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the executable instructions to implement the method of any one of claims 1-4.

Images