Multi-station target deduplication method, device, equipment, storage medium and program product

Abstract

The application discloses a multi-camera target deduplication method and device, equipment, a storage medium and a program product, and relates to the technical field of computer vision. The method comprises the following steps: acquiring a panoramic image and at least one close-up image; determining an overlapping area of the panoramic image and the close-up image; detecting a first target person in the overlapping area in the panoramic image, and detecting a second target person in the close-up image; performing similarity matching on a first spatial distribution relationship of the first target person in the overlapping area and a second spatial distribution relationship of the second target person in the close-up image to obtain a similarity matching result; and determining a repeated person and removing the repeated person from an output picture according to the similarity matching result, wherein the output picture is the panoramic image or the close-up image in which the repeated person is located. The application meets the real-time processing requirement of an edge device while ensuring the deduplication accuracy, and realizes accurate identification and removal of repeated targets in multiple video streams under the condition of limited computing power.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a multi-camera target deduplication method, a multi-camera target deduplication device, a multi-camera target deduplication equipment, a storage medium and a computer program product. BACKGROUND

[0002] In a traditional video monitoring system, each camera operates independently to detect and track the people in the field of view. When multiple cameras exist in overlapping or adjacent coverage areas, the same person can be recorded in multiple cameras, resulting in problems such as data redundancy, statistical distortion, and resource waste.

[0003] To address this problem, existing technologies have developed simple matching based on timestamps and location information, matching based on color histograms and appearance features, and person re-identification (Re-ID) combining hand-designed features and metric learning. In particular, with the development of deep learning, Re-ID technology has become the core solution for cross-camera person deduplication, which learns discriminative feature representations of people through deep neural networks to achieve identity matching under different perspectives, lighting, and poses.

[0004] However, when applying the above general solution to a conference scenario, there are dual challenges of insufficient feature discrimination and high computational complexity, which limit the deduplication accuracy. Specifically, a conference scenario usually has wide-angle panoramic and multiple focal cameras, resulting in a high demand for the computing power of edge devices. Limited AI computing power makes it difficult to support real-time and deep AI analysis, resulting in low efficiency of existing complex Re-ID models in actual deployment, making it difficult to meet real-time processing requirements. Secondly, the behavior patterns of conference participants are special, as most of them are in a seated position and not always facing the camera, which makes the FaceID (Facial Identification) technology, which relies heavily on facial clarity and frontal angle, ineffective and has a very low recognition probability. Furthermore, conference participants often wear similar and uniform clothing, and their activity range is limited, resulting in a significant reduction in the discrimination of their appearance. The discriminative features such as clothing texture and body contour relied on by traditional Re-ID technology have significantly reduced discrimination ability in this scenario, resulting in a significant decrease in deduplication accuracy. SUMMARY

[0005] The main purpose of the present application is to provide a multi-camera target deduplication method, a multi-camera target deduplication device, a multi-camera target deduplication equipment, a storage medium and a computer program product, aiming to solve the technical problem of low deduplication accuracy in a conference scenario.

[0006] To achieve the above object, the application provides a multi-position target deduplication method, which comprises the following steps:

[0007] obtaining a panoramic image and at least one close-up image;

[0008] determining an overlapping area of the panoramic image and the close-up image, detecting a first target person in the overlapping area in the panoramic image, and detecting a second target person in the close-up image;

[0009] performing similarity matching on a first spatial distribution relationship of the first target person in the overlapping area and a second spatial distribution relationship of the second target person in the close-up image to obtain a similarity matching result;

[0010] determining a duplicate person according to the similarity matching result and removing the duplicate person from an output image, wherein the output image is the panoramic image or the close-up image in which the duplicate person is located.

[0011] In an embodiment, the step of performing similarity matching on the first spatial distribution relationship of the first target person in the overlapping area and the second spatial distribution relationship of the second target person in the close-up image to obtain a similarity matching result comprises:

[0012] performing similarity matching on a first feature vector of the first target person in the overlapping area and a second feature vector of the second target person in the close-up image to obtain a similarity matching result, wherein the first feature vector is generated based on a relative position relationship of the first target person in the overlapping area, and the second feature vector is generated based on a relative position relationship of the second target person in the close-up image.

[0013] In an embodiment, before the step of performing similarity matching on the first feature vector of the first target person in the overlapping area and the second feature vector of the second target person in the close-up image to obtain a similarity matching result, the method further comprises:

[0014] taking a position of the first target person in the panoramic image as a first origin to establish a first coordinate system, calculating a first relative position relationship of the remaining first target persons in the overlapping area relative to the first origin, and generating a first feature vector based on the first relative position relationship; wherein the first relative position relationship comprises a quadrant relationship and an included angle relationship, and the first feature vector comprises a quadrant vector and an angle vector.

[0015] The position of the second target person in the close-up image is taken as a second origin to establish a second coordinate system, second relative position relationships of the remaining second target persons in the close-up image relative to the second origin are calculated, and a second feature vector is generated based on the second relative position relationships; wherein the second relative position relationships include a quadrant relationship and an included angle relationship, and the second feature vector includes a quadrant vector and an angle vector.

[0016] In an embodiment, the step of generating the first feature vector based on the first relative position relationships comprises:

[0017] determining first quadrant numbers of the remaining first target persons in the first coordinate system in the overlapping area, and generating first quadrant vectors of the first target persons according to the first quadrant numbers;

[0018] The step of generating the second feature vector based on the second relative position relationships comprises:

[0019] determining second quadrant numbers of the remaining second target persons in the second coordinate system in the close-up image, and generating second quadrant vectors of the second target persons according to the second quadrant numbers;

[0020] The step of performing similarity matching between the first feature vector of the first target person generated based on the relative position relationships in the overlapping area and the second feature vector of the second target person generated based on the relative position relationships in the close-up image to obtain a similarity matching result comprises:

[0021] by traversing the first quadrant vector and the second quadrant vector, the number of matching quadrants with the same quadrant number is counted, and the similarity matching result is determined based on the number of matching quadrants, wherein the more the number of matching quadrants is, the higher the similarity is.

[0022] In an embodiment, the step of generating the first feature vector based on the first relative position relationships further comprises:

[0023] determining a first target included angle of the remaining first target persons in the first coordinate system relative to the first origin in the overlapping area, and generating a first angle vector of the first target person according to an angle value of the first target included angle;

[0024] The step of generating the second feature vector based on the second relative position relationships further comprises:

[0025] determining a second target included angle of the remaining second target persons in the second coordinate system relative to the second origin in the close-up image, and generating a second angle vector of the second target person according to an angle value of the second target included angle;

[0026] The step of performing similarity matching on the first spatial distribution relationship of the first target person in the overlapping area and the second spatial distribution relationship of the second target person in the close-up image to obtain a similarity matching result further includes:

[0027] The step of performing similarity matching on the first spatial distribution relationship of the first target person in the overlapping area and the second spatial distribution relationship of the second target person in the close-up image to obtain a similarity matching result further includes:

[0028] In an embodiment, after the step of performing similarity matching on the first spatial distribution relationship of the first target person in the overlapping area and the second spatial distribution relationship of the second target person in the close-up image to obtain a similarity matching result, the method further includes:

[0029] Obtaining a first repeated person matching pair set determined by the quadrant vector;

[0030] Obtaining a second repeated person matching pair set determined by the angle vector;

[0031] Taking a union of the first repeated person matching pair set and the second repeated person matching pair set to determine a final repeated person.

[0032] In addition, to achieve the above object, the application further provides a multi-camera target deduplication device, which comprises:

[0033] An acquisition module is configured to acquire a panoramic image and at least one close-up image;

[0034] A detection module is configured to determine an overlapping area of the panoramic image and the close-up image, detect a first target person in the overlapping area in the panoramic image, and detect a second target person in the close-up image;

[0035] A matching module is configured to perform similarity matching on a first spatial distribution relationship of the first target person in the overlapping area and a second spatial distribution relationship of the second target person in the close-up image to obtain a similarity matching result;

[0036] A deduplication module is configured to determine a repeated person according to the similarity matching result and remove the repeated person from an output image, wherein the output image is a panoramic image or a close-up image in which the repeated person is located.

[0037] In addition, to achieve the above object, the present application also provides a multi-camera target deduplication device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the multi-camera target deduplication method as described above.

[0038] In addition, to achieve the above object, the present application also provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the multi-camera target deduplication method as described above.

[0039] In addition, to achieve the above object, the present application also provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the multi-camera target deduplication method as described above.

[0040] The one or more technical solutions provided by the present application have at least the following technical effects:

[0041] Since the existing general deduplication scheme has the challenges of insufficient feature distinction and high computational complexity when applied to a conference scene, the deduplication accuracy is low.

[0042] The present application significantly reduces the computational load by detecting only the target in the overlapping area corresponding to the close-up image in the panoramic image, rather than processing the entire panoramic image, thereby alleviating the real-time processing pressure caused by limited AI computing power of the device. Secondly, by using the feature vector generated based on the spatial distribution relationship for matching, the appearance features of the characters, which are easily affected by the similarity of the clothes, are no longer relied on, effectively overcoming the problem of insufficient feature distinction in the conference scene. The spatial distribution relationship between the characters is relatively stable, and even in the scene where all the characters are in a sitting position and have non-frontal faces, the stable discrimination ability can still be maintained. Finally, the repeated characters are determined by spatial relationship similarity matching and deduplicated in the final output picture, establishing a matching mechanism that does not rely on face recognition and has high computational efficiency, thereby ensuring the deduplication accuracy while meeting the real-time processing requirements of the edge device, and realizing the accurate identification and removal of repeated targets in multiple video streams under the condition of limited computing power. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings are only for the purpose of illustrating the embodiments of the present application, and for the person skilled in the art, other drawings can also be obtained without creative labor.

[0045] Figure 1 The first schematic diagram provided for an embodiment of the multi-camera target deduplication method of the present application;

[0046] Figure 2 The second schematic diagram provided for an embodiment of the multi-camera target deduplication method of the present application;

[0047] Figure 3 The flowchart provided for an embodiment of the multi-camera target deduplication method of the present application;

[0048] Figure 4 The third schematic diagram provided for an embodiment of the multi-camera target deduplication method of the present application;

[0049] Figure 5 The fourth schematic diagram provided for an embodiment of the multi-camera target deduplication method of the present application;

[0050] Figure 6 The fifth schematic diagram provided for an embodiment of the multi-camera target deduplication method of the present application;

[0051] Figure 7 The sixth schematic diagram provided for an embodiment of the multi-camera target deduplication method of the present application;

[0052] Figure 8 The module structure schematic diagram of the multi-camera target deduplication device of the present application;

[0053] Figure 9 The device structure schematic diagram of the hardware running environment involved in the multi-camera target deduplication method of the present application.

[0054] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0055] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0056] In order to better understand the technical solutions of the present application, the following will be described in detail with reference to the drawings and specific embodiments in the specification.

[0057] To cope with the problem that the same person can be recorded in multiple cameras when multiple cameras exist overlapping or adjacent coverage areas, resulting in data redundancy, statistical distortion, resource waste and other problems. The existing technology has developed simple matching based on timestamp and location information, matching based on color histogram and appearance features, and person re-identification (Re-ID) combining hand-designed features and metric learning.

[0058] Among them, the simple matching method based on timestamp and location information assumes that the person appears in adjacent cameras at close to the same time and with a reasonable moving path. However, the accuracy rate significantly decreases in complex scene topologies or high-density crowd environments. The matching method based on color histogram and appearance features compares the low-level visual features such as clothing color and contour of the person. However, this method is easily affected by changes in lighting, occlusion interference and perspective transformation, and has poor robustness. The person re-identification method combining hand-designed features and metric learning, such as Histogram of Oriented Gradients (HOG) and Local Binary Pattern (LBP) combined with metric learning technology, has improved the accuracy of person re-identification Re-ID to some extent, but still cannot overcome the inherent difficulties such as large perspective difference and inconsistent camera parameters.

[0059] With the development of deep learning technology, person re-identification Re-ID based on deep neural network has become the core technology for multi-camera person deduplication. It extracts the deep feature embedding vector of the person through a pre-trained model and calculates the feature similarity between different cameras to realize cross-camera identity matching. However, this person re-identification still faces challenges such as high real-time processing computing power requirement, difficulty in deploying edge devices, high cost of training data labeling, weak cross-domain generalization ability, and low deduplication accuracy in conference scenarios due to insufficient feature discrimination and high computational complexity.

[0060] The application significantly reduces the amount of calculation by detecting only the target in the overlapping area corresponding to the close-up image in the panoramic image instead of processing the entire panoramic image, thereby relieving the real-time processing pressure caused by the limited AI computing power of the device. Secondly, by using the feature vector generated based on the spatial distribution relationship for matching, the application no longer relies on the appearance features of the characters which are easily affected by the similarity of the clothes, effectively overcoming the problem of insufficient feature discrimination in the conference scene. The spatial distribution relationship between the characters is relatively stable, and even in the scene where all the characters are in a sitting position and have non-frontal faces, the application can still maintain stable discrimination ability. Finally, the application determines the repeated characters by spatial relationship similarity matching and removes the repeated characters in the final output picture, thereby establishing a matching mechanism that does not rely on face recognition and has high computing efficiency, so as to ensure the accuracy of the removal while meeting the real-time processing needs of the edge device, and realize the accurate identification and removal of repeated targets in multiple video streams under the condition of limited computing power.

[0061] As shown in Figure 1 , Figure 1 A scenario applicable to the application is that in a conference room, there are a sound box, a display screen, and a camera represented by reference numeral 100 placed at the top of the large screen for capturing panoramic images and multiple close-up images of the conference scene. As shown in Figure 2 , the number and distribution of the cameras can include one panoramic camera 14 and three close-up cameras 11, 12, and 13; 15 represents the camera field of view of the close-up camera; and 16 represents the camera field of view of the panoramic camera.

[0062] It should be noted that the execution subject of the embodiment can be a multi-camera target deduplication device such as an edge device, a computing service device having data processing, network communication, and program running functions, for example, a camera, a tablet computer, a personal computer, a mobile phone, or the like, or an electronic device or a processor capable of realizing the above functions. The following will take the multi-camera target deduplication device (hereinafter referred to as the deduplication device) as an example to describe the embodiment and the following embodiments.

[0063] Based on this, the embodiment of the application provides a multi-camera target deduplication method, as shown in Figure 3 , Figure 3 is a flowchart of an embodiment of the multi-camera target deduplication method of the application.

[0064] In the embodiment, the multi-camera target deduplication method includes steps S10-S40:

[0065] Step S10, acquiring a panoramic image and at least one close-up image;

[0066] In an embodiment, the panoramic image and the at least one close-up image are acquired by deploying multiple cameras in the conference scene.

[0067] Optionally, one wide-angle fixed-focus panoramic camera is responsible for capturing the overall environment, providing a wide-angle view covering the entire conference area, while multiple fixed-focus close-up cameras focus on specific areas, such as the faces or upper bodies of the participants, to obtain detailed close-up pictures.

[0068] Illustratively, the panoramic image or close-up image can come from a real-time video stream or a pre-recorded video frame, ensuring the timeliness and integrity of the data source.

[0069] It can be understood that such a multi-camera configuration aims to comprehensively cover the conference scene and create an immersive experience for remote participants, but at the same time introduces the risk of image overlap and redundant targets, so an effective deduplication mechanism is needed to avoid the same person appearing repeatedly in multiple shots.

[0070] Step S20, determining the overlapping area of the panoramic image and the close-up image, detecting a first target person in the overlapping area in the panoramic image, and detecting a second target person in the close-up image;

[0071] It should be noted that in order to avoid the same person appearing repeatedly in multiple shots, multiple people in different shots need to be found, i.e. the overlapping area between the panoramic image and the close-up image needs to be determined, and the target person needs to be detected in these areas. Among them, the overlapping area is the repeated field of view of the cameras that shoot the panoramic image and the close-up image.

[0072] Optionally, the overlapping area can be identified through the calibration parameters or geometric transformation of the cameras, for example, based on the mapping of the perspective of the close-up camera in the panoramic image to the corresponding range.

[0073] In a specific embodiment, the target person detection can use a target detection algorithm, such as YOLO (You Only Look Once single multi-frame detection) or Faster R-CNN (Faster Region-based Convolutional Neural Network, region-based fast convolutional neural network), to detect the first target person in the overlapping area of the panoramic image and the second target person in the close-up image.

[0074] Illustratively, the first target person can include one or more participants, and the second target person is the focal individual in the close-up shot, which can also include one or more participants. It can be understood that this step is the basis for deduplication, identifying candidate targets that may appear repeatedly.

[0075] As Figure 4As shown, label 101 represents a close-up image, label 102 represents a target person photographed by the camera, which is referred to as a second target person in the close-up image; label 103 represents a panoramic image, and the target person in the panoramic image is referred to as a first target person; and label 104 represents an overlapping area of the panoramic image and the close-up image. The determination of the overlapping area can be artificially set, or can be determined according to the target person, such as the location, the distribution between persons, face recognition, etc., without limitation.

[0076] In step S30, the first spatial distribution relationship of the first target person in the overlapping area and the second spatial distribution relationship of the second target person in the close-up image are similarity matched to obtain a similarity matching result.

[0077] It should be noted that the similarity matching of the first spatial distribution relationship of the first target person in the overlapping area and the second spatial distribution relationship of the second target person in the close-up image is used to evaluate whether it is the same person.

[0078] In the specific embodiment, the quadrant method or the angle method can be used to realize the matching; for example, in the quadrant method, a Cartesian coordinate system is established with one target person in the panoramic image as the origin, the quadrant positions of other target persons relative to the origin are calculated to form a quadrant vector, and similar operations are performed in the close-up image, and then the matching result is obtained by comparing the similarity of the vectors.

[0079] Alternatively, the angle method is based on the angle vector between the target persons to calculate the difference between the angle and the preset threshold. It can be understood that these methods use relative position information, reduce the dependence on appearance features, and thus realize efficient deduplication on devices with limited computing power, and can be applied to the challenge of similar dress and large posture changes in a conference scene.

[0080] In a feasible embodiment, step S30 includes step S301: similarity matching a first feature vector of the first target person in the overlapping area based on the relative position relationship with a second feature vector of the second target person in the close-up image based on the relative position relationship to obtain a similarity matching result.

[0081] It should be noted that the relative position relationship can be quantified by the quadrant method or the angle method as described above, for example, the orientation or angle of other targets relative to a certain target as a reference origin is encoded into an ordered numerical sequence, thereby forming a feature vector of the reference target.

[0082] Exemplarily, the first feature vector of a person in the panoramic image can contain its quadrant relationship with all other persons in the vicinity, while the second feature vector of a person in the close-up image encodes its relative position with other persons in the close-up picture. Then the similarity between the feature vectors is calculated, for example, by comparing the matching degree of the vector sequences or calculating the distance between the vectors.

[0083] It can be understood that the method of abstracting the spatial distribution relationship into the feature vector effectively captures the topological structure between the persons, reduces the dependence on the appearance features of the individuals, and thus makes the matching process more robust to changes in illumination, dress and posture, and is suitable for running on edge devices with limited computing resources, such as mobile phones or cameras.

[0084] In the embodiment, based on the matching mechanism of the relative position topological relationship, the complex spatial distribution relationship is abstracted into a structured feature vector for matching by using the stable geometric relationship between the persons instead of the variable appearance features, so that the person deduplication in a multi-camera scene can be realized without relying on high-power AI analysis, and thus the problems of similar dress, posture change and non-frontal face in a conference scene can be effectively overcome. Even if the conference participants are all in formal dress and in a sitting posture, the quadrant or angle vector constituted by the relative positions of the persons in the picture still has a high degree of distinguishability. The calculation complexity is transferred from the feature extraction relying on a deep model to the lightweight vector comparison, which significantly reduces the requirement for the computing power of the edge device, can perform real-time deduplication processing on a wide-angle panoramic video stream and multiple close-up video streams, achieves a balance between the calculation efficiency and practicability, and practically solves the pain point of repeated display of persons in a multi-camera conference system.

[0085] In step S40, a repeated person is determined according to the similarity matching result, and the repeated person is removed from the output picture, wherein the output picture is a panoramic image or a close-up image in which the repeated person is located.

[0086] It should be noted that the repeated person is determined according to the similarity matching result, and the repeated person is removed from the output picture, wherein the output picture can be a panoramic image or a close-up image.

[0087] In the specific embodiment, the matching results of the quadrant method and the angle method are combined to identify the target corresponding to the same person in the panoramic image and the close-up image, and then the redundant instances are removed from the picture by using image processing techniques such as masking or cropping. Exemplarily, if a panoramic person and a close-up person have the highest score in the spatial distribution matching, the panoramic person and the close-up person are regarded as repeated persons, and the clear person in the close-up picture can be retained while the corresponding person in the panoramic image is hidden, or the corresponding person in the panoramic image can be retained while the corresponding person in the close-up picture is hidden.

[0088] It can be understood that this step ensures that each participant appears only once in the final output multi-person close-up gallery picture, avoids redundancy, and improves the immersion of the remote meeting experience.

[0089] The embodiment provides a multi-camera target deduplication method. The application detects only targets in an overlapping area corresponding to a close-up image in a panoramic image, instead of processing the entire panoramic image, thereby significantly reducing the calculation amount and relieving the real-time processing pressure caused by limited AI computing power of a device. Secondly, the feature vector generated based on the spatial distribution relationship is used for matching, instead of relying on the appearance features of the person which are easily affected by the similarity of the clothes, thereby effectively overcoming the problem of insufficient feature discrimination in a conference scene. The spatial distribution relationship between the persons is relatively stable, and even in a scene where all persons are in a sitting posture and do not have a front face, the stable discrimination ability can be maintained. Finally, the repeated persons are determined through spatial relationship similarity matching and are deduplicated in the final output picture, thereby establishing a matching mechanism which does not depend on face recognition and has high calculation efficiency, so that the real-time processing demand of an edge device is met while the deduplication accuracy is ensured, and the accurate identification and removal of repeated targets in multiple video streams under the condition of limited computing power are realized.

[0090] Based on the above-mentioned embodiments, in another embodiment of the application, the same or similar contents as the above-mentioned embodiments can be referred to the above introduction, and will not be described in detail. On this basis, before step S301, the multi-camera target deduplication method further includes steps S31-S32:

[0091] In step S31, the position of the first target person in the panoramic image is taken as a first origin to establish a first coordinate system, the first relative position relationship of the remaining first target persons in the overlapping area relative to the first origin is calculated, and a first feature vector is generated based on the first relative position relationship; wherein the first relative position relationship includes a quadrant relationship and an included angle relationship, and the first feature vector includes a quadrant vector and an angle vector.

[0092] It should be noted that the position of the first target person currently being processed in the panoramic image is set as the first origin, and the first coordinate system is established based on the first origin. Alternatively, the first coordinate system can be a Cartesian coordinate system.

[0093] In the specific embodiment, when the first target person is multiple, one of the first target persons is selected as the first origin, and the first relative position relationship of all other first target persons in the overlapping area relative to the first origin is calculated. The relative position relationship can be a quadrant relationship, that is, which quadrant of the coordinate system the other target is located in; or an included angle relationship, that is, the included angle formed by the connection line of the other target and the positive direction of the horizontal axis or the vertical axis of the coordinate system (the included angle with which coordinate axis is not limited).

[0094] Exemplarily, the data obtained from the quadrant relationship or the included angle relationship is sorted and ordered to finally generate the first feature vector containing the ordered quadrant vector and the angle vector. It can be understood that this step digitizes the spatial topological structure around each target person and aims to construct the spatial feature representation of each first target person in the panoramic image.

[0095] In step S32, the position of the second target person in the close-up image is set as a second origin to establish a second coordinate system, the second relative position relationship of the remaining second target persons in the close-up image relative to the second origin is calculated, and a second feature vector is generated based on the second relative position relationship; wherein the second relative position relationship includes the quadrant relationship and the included angle relationship, and the second feature vector includes the quadrant vector and the angle vector.

[0096] It should be noted that the position of the second target person currently being processed in the close-up image is set as the second origin, and the second coordinate system is established, which aims to generate the corresponding spatial feature representation for each second target person in the close-up image. Optionally, the second coordinate system can be a Cartesian coordinate system.

[0097] In the specific embodiment, when there are multiple second target persons, one of the second target persons is selected as the second origin, and the second relative position relationship of all other second target persons in the close-up image relative to this second origin is calculated.

[0098] Optionally, the second relative position relationship also includes the quadrant relationship and the included angle relationship, consistent with the processing manner of the overlapping area in the panoramic image. Exemplarily, the second feature vector is generated based on the quadrant relationship or the included angle relationship, and also contains the ordered quadrant vector and the angle vector.

[0099] It can be understood that by establishing the local coordinate system with the current target person as the origin in the panoramic image and the close-up image and generating the feature vector, the spatial distribution of the person under different camera angles is unified to a comparable mathematical representation, which lays a foundation for subsequent similarity matching.

[0100] In the embodiment, the relative position relationship in the visual space is creatively converted into a quantifiable mathematical representation by constructing a local coordinate system with each target person as the origin in the panoramic and close-up images respectively, and generating a characteristic vector of the quadrant relationship or the included angle relationship. It can be understood that this feature expression method based on local topological relationship enables the same person under different angles and different focal lengths to be associated and matched through the stable spatial structure relationship with the surrounding people. This method, which does not rely on pixel-level appearance comparison, fundamentally overcomes the recognition difficulties caused by changes in lighting, similar clothing and posture differences, while controlling the computational complexity at the geometric operation level, significantly reducing the dependence on edge device AI computing power, and providing a feasible technical solution for realizing real-time and efficient deduplication of multi-path video streams.

[0101] In a possible implementation, the step S31 comprises:

[0102] determining a first quadrant number in which the remaining first target person in the overlap region is located in the first coordinate system, and generating a first quadrant vector of the first target person according to the first quadrant number;

[0103] It should be noted that a quadrant feature representation is constructed for each first target person in the panoramic image. The position of the currently processed first target person is taken as the origin of the first coordinate system, and then the positions of all other first target persons in the overlap region relative to the origin are determined and mapped to specific quadrant numbers.

[0104] Exemplarily, in the Cartesian coordinate system, the other target persons will be divided into one of the first to fourth quadrants according to their horizontal and vertical coordinate relationships relative to the origin, and obtain the corresponding quadrant numbers. Subsequently, these quadrant numbers are sorted and arranged according to a preset rule (from small to large or from large to small), and finally an ordered first quadrant vector is generated, which can represent the spatial distribution pattern of the surrounding people of the origin person.

[0105] It can be understood that this method of quantifying the relative spatial position into a sequence of quadrant numbers effectively captures the topological structure relationship between the people.

[0106] The step S32 comprises:

[0107] determining a second quadrant number in which the remaining second target person in the close-up image is located in the second coordinate system, and generating a second quadrant vector of the second target person according to the second quadrant number;

[0108] It should be noted that on the side of the close-up image, a symmetric processing procedure is performed to generate the quadrant feature representation of the second target person. This step takes the position of the currently processed second target person as the origin of the second coordinate system, determines the quadrant in which the remaining second target person in the close-up image is located relative to the origin, and obtains the corresponding second quadrant number.

[0109] Exemplarily, the second quadrant numbers are arranged into an ordered sequence to form the second quadrant vector of the currently processed second target person in the close-up image.

[0110] It can be understood that through this unified quantization method, the spatial layout of the person in the panorama and the close-up image is converted into mathematical vectors with the same structure and can be directly compared.

[0111] The step of performing similarity matching between the first feature vector generated based on the relative position relationship of the first target person in the overlapping area and the second feature vector generated based on the relative position relationship of the second target person in the close-up image includes:

[0112] By traversing the first quadrant vector and the second quadrant vector, the number of matching quadrants with the same quadrant number is counted, and the similarity matching result is determined based on the number of matching quadrants, wherein the more the number of matching quadrants, the higher the similarity.

[0113] It should be noted that after the construction of the quadrant vector is completed, the similarity matching is performed by traversing and comparing the first quadrant vector and the second quadrant vector.

[0114] In the specific embodiment, the quadrant numbers at the same index positions in the two quadrant vectors are compared bit by bit, and the number of units with consistent numbers is counted, which is the number of matching quadrants. Alternatively, the number of matching quadrants is directly used as a data indicator for measuring similarity, and the larger the value is, the more similar the relative spatial environment of the two target persons in their respective images, i.e., the higher the similarity.

[0115] Exemplarily, if the quadrant vector of a person A in the panorama and the quadrant vector of a person B in the close-up image have the same quadrant number at most positions, it is determined that A and B have a highly similar spatial relationship, i.e., it can be determined that it is the image of the same person under different cameras.

[0116] It can be understood that this method based on relative position quadrant matching skillfully avoids many difficulties of directly relying on the appearance features of the person for recognition, and provides a lightweight and robust technical solution for efficient and accurate multi-camera target deduplication.

[0117] Exemplarily, the following code can be used when traversing the quadrant vector:

[0118] Match_list = []; / / Initialize an empty list named Match_list. This list is used to store the matching degree between the first target person i in the panoramic image and the second target person j in each close-up image.

[0119] foriin[A1,A2,A3]; / / Iterate through the first quadrant vector [A1, A2, A3] of a specific first target person in the currently being processed panoramic image, where A1, A2, A3 are specific quadrant numbers (e.g., 1, 2, 3, 4, representing the four quadrants respectively).

[0120] Match;

[0121] for j in [E1,E2]: / / Iterate through the second quadrant vector [E1, E2] of a specific second target person in a close-up image that is currently being processed and is used for comparison with the first target person i in the panoramic image, where E1 and E2 are quadrant numbers.

[0122] If panoramic and close-up vectors with the same index belong to the same quadrant; / / In the current loop, are the quadrant numbers of the same index positions of the panoramic vector [A1, A2, A3] and the close-up vector [E1, E2] equal? ​​That is, when the outer loop index is k (e.g., k=0 corresponds to A1, k=1 corresponds to A2), and the inner loop index is also k (e.g., k=0 corresponds to E1, k=1 corresponds to E2), it is considered "same index". Compare A1 and E1, then compare A2 and E2. Vector [A1, A2, A3] is longer than [E1, E2], so only the effective length (i.e., length 2) of the two vectors is compared. Vector A3 is not compared because there is no corresponding third element in the second quadrant vector.

[0123] Match += 1; / / If the same index quadrant is the same, then increment the Match value by 1.

[0124] Match_list.append(Match); / / After the inner loop finishes, add the total match score Match between the first target person i and the second target person j in the close-up image to the Match_list list.

[0125] Additionally, it should be noted that the number of elements in the quadrant vector is determined by the number of target figures. In this application, three elements for the panoramic image and two elements for the close-up image are used as an example. Specifically, the number of elements can be adjusted according to the target figures detected in the actual image, and the same index will be compared during matching.

[0126] In the embodiment, by converting the spatial distribution of the person in the panoramic view and the close-up view into quantifiable quadrant vectors, and using a simple traversal comparison algorithm, an efficient person deduplication solution is constructed. Specifically, by establishing a local coordinate system for each target person and generating an ordered quadrant number sequence, the complex spatial topological relationship is converted into a standardized mathematical expression; then by comparing the quadrant numbers at the same index position in the two vectors, the consistency of the spatial distribution of the person in different shots can be accurately evaluated.

[0127] This matching mechanism based on relative position relationship fundamentally avoids the problems of illumination change and similar clothing faced by traditional methods relying on appearance features, and its calculation process only involves simple logical judgment and counting operations, which has extremely high execution efficiency. This innovative method of converting spatial topological relationship into computable vectors not only ensures the accuracy of matching, but also enables real-time multi-channel video stream processing on edge devices with limited computing power, ultimately effectively solving the target deduplication problem in multi-camera systems at a low computational cost.

[0128] Figure 5 Figure (a) in the above is a schematic diagram of obtaining a quadrant vector of a first target person in the overlapping area of the panoramic image and the close-up image; specifically, point A is selected as the first origin, the quadrants of points B, C and D relative to point A in the first coordinate system are calculated, and they are arranged in order from small to large according to the quadrant number; according to the quadrant reference diagram, the first quadrant vector of point A is A(1, 3, 4); by analogy, the first quadrant vector of point B is B(3, 3, 4); the first quadrant vector of point C is C(2, 2, 3); and the first quadrant vector of point D is D(1, 1, 1).

[0129] Figure 5 Figure (b) in the above is a schematic diagram of obtaining a quadrant vector of a second target person in the close-up image; specifically, point E is selected as the second origin, the quadrants of points F and G relative to point E in the second coordinate system are calculated, and they are arranged in order from small to large according to the quadrant number; according to the quadrant reference diagram, the second quadrant vector of point E is E(1, 4); by analogy, the second quadrant vector of point F is F(3, 4); and the second quadrant vector of point G is G(2, 2).

[0130] Then, the matching between the point A in the overlapping area of the panoramic image and the points E, F and G in the close-up image is performed. The matching method is as follows: the first two digits of (1, 3, 4) are compared with (1, 4), (3, 4) and (2, 2) respectively to see if they are the same. For example, if one digit is the same compared with (1, 4), it is recorded as 1; if there is no same digit compared with (3, 4) and (2, 2), it is recorded as 0. Finally, the Match_list of the point A is [1, 0, 0]. The close-up target corresponding to the maximum matching number is taken as the matching target in the panoramic image. Finally, the matching target of the point A in the panoramic image is the point E. The matching of all the points is performed in turn.

[0131] In another possible implementation, the step S31 further includes:

[0132] determining a first target angle of each of the remaining first target persons in the overlapping area relative to the first origin in the first coordinate system, and generating a first angle vector of the first target person according to an angle value of the first target angle;

[0133] It should be noted that the generation method of the angle vector is introduced when the spatial features are constructed.

[0134] On the basis of the established first coordinate system, the angle formed by the connecting line of each of the remaining first target persons in the overlapping area relative to the first origin and the positive direction of the horizontal axis of the coordinate system is calculated. These angles are defined as the first target angles.

[0135] Exemplarily, the specific angle values of these angles can be calculated by using the inverse trigonometric function and the like. Subsequently, the angle values are sorted according to a preset rule, such as the order of the angles from small to large, to finally form the first angle vector of the first target person.

[0136] It can be understood that this generation method of the angle vector describes the orientation relationship between the origin person and the other persons from another geometric dimension, and provides more abundant spatial information for subsequent matching.

[0137] The step S32 further includes:

[0138] determining a second target angle of each of the remaining second target persons in the close-up image relative to the second origin in the second coordinate system, and generating a second angle vector of the second target person according to an angle value of the second target angle;

[0139] It should be noted that correspondingly, the construction of the angle vector is also performed on the close-up image side. This process can be parallel or first complete the panoramic image and then complete the close-up image, which is not limited. In the second coordinate system, the second target angles of the connecting lines of the remaining second target persons in the close-up image relative to the second origin and the positive direction of the horizontal axis are determined, and the angle values thereof are obtained.

[0140] In the specific implementation, the angle values are also sorted to generate a second angle vector of the second target person. Optionally, the sorting rule is consistent with the panorama side to ensure that the two vectors are comparable.

[0141] It can be understood that by synchronously generating angle vectors on both panorama and close-up sides, another complementary feature describing the spatial distribution of the person is obtained, which together with the quadrant vector constitutes a relative position description system.

[0142] The step of performing similarity matching between the first feature vector of the first target person generated based on the relative position relationship in the overlapping area and the second feature vector of the second target person generated based on the relative position relationship in the close-up image to obtain a similarity matching result further comprises:

[0143] The first angle vector and the second angle vector are traversed, the number of angles whose absolute difference of angle values at corresponding positions is less than a preset angle threshold is counted, and the similarity matching result is determined based on the number of angles, wherein the more the number of angles, the higher the similarity.

[0144] It should be noted that when performing similarity matching, the first angle vector and the second angle vector are compared by traversal.

[0145] In the specific implementation, the angle values of the two angle vectors at the same sequence positions are compared one by one, and the size of the absolute difference between them is calculated. Exemplarily, the number of corresponding positions whose absolute difference is less than a preset angle threshold is counted, and this number is the matching angle number. Optionally, the preset angle threshold can be adjusted according to the actual application scenario, for example, set to five degrees. The more the matching angle number, the more consistent the angle relationship formed by the two target persons with other persons in their respective pictures, that is, the more similar their spatial distribution patterns, and the higher the similarity.

[0146] It can be understood that this matching strategy based on angle tolerance, combined with the quadrant matching method, forms a multi-feature fusion decision mechanism, which significantly improves the accuracy and robustness of the person deduplication decision, especially in the conference scene where the appearance features of the persons are similar or change greatly. This matching method relying on stable spatial relationship shows significant advantages.

[0147] The following code can be used when traversing the angle vector:

[0148] Match_list = []; / / Initialize an empty list named Match_list (matching list). This list is used to store the matching degree between the first target person i in the panorama image and the second target person j in each close-up image.

[0149] for i in [B1,B2,B3]; / / Traverse the first angle vector [B1,B2,B3] of a specific first target person in a panoramic image currently being processed. B1, B2, and B3 are specific angle values (e.g., 0° - 360°).

[0150] Match;

[0151] for j in [F1,F2]: / / Traverse the second angle vector [F1,F2] of a specific second target person in a close-up image currently being processed for comparison with the first target person i in the panoramic image, where F1 and F2 are specific angle values.

[0152] If the included angle between the panoramic and close-up vectors with the same subscript < T / / In the current loop, calculate the absolute difference between the angle values represented by the elements of the first angle vector in the panoramic image and the corresponding elements of the second angle vector in the close-up image.

[0153] Match += 1; / / If this absolute difference is less than the preset angle threshold T, then increment the value of the match degree Match by 1.

[0154] Match_list.append(Match); / / After the inner loop ends, add the total match degree Match between the first target person i in the panoramic image and the second target person j in the close-up image obtained from this calculation to the Match_list list.

[0155] In addition, it should be noted that the number of elements in the angle vector is determined by the number of target persons. In this application, three elements for the panoramic image and two elements for the close-up image are taken as an example. Specifically, it can be adjusted according to the target persons detected in the actual image, and the same subscripts will also be compared during matching.

[0156] In this embodiment, by introducing the angle vector matching method, the included angle values between target persons are calculated and sorted to generate an angle vector, thereby further capturing spatial orientation information based on the direction relationship. This matching strategy based on angle tolerance effectively complements the quadrant matching method: the quadrant method grasps the macroscopic layout, and the angle method controls the microscopic orientation. It can be understood that this dual verification mechanism significantly enhances the ability of the deduplication device to handle complex scenarios. Even in the case of changes in the pose of a person, differences in viewing angles, or partial occlusion, stable matching performance can be maintained through at least one spatial relationship feature. Finally, this solution achieves high-precision real-time deduplication of multiple video streams on resource-constrained edge devices while maintaining a low computational complexity.

[0157] As Figure 6 shown, Figure 6The (a) figure in the (a) figure in the panoramic image is a schematic diagram of the first target person in the overlapping area of the panoramic image and the close-up image when the angle vector is obtained. Specifically, point A is selected as the second origin, the included angle values of points B, C and D with the positive direction of the horizontal axis in the first coordinate system are calculated, and the included angle values are arranged in ascending order. For example, the first angle vector of point A is A(a2, a1, a3), such as (15°, 30°, 120°). Similarly, the first angle vector of point B, the first angle vector of point C and the first angle vector of point D can be obtained.

[0158] Figure 6 The (b) figure in the (b) figure in the close-up image is a schematic diagram of the second target person when the angle vector is obtained. Specifically, point E is selected as the second origin, the included angle values of points F and G with the positive direction of the horizontal axis in the second coordinate system are calculated, and the included angle values are arranged in ascending order. For example, the second angle vector of point E is E(b2, b1), such as (15°, 30°). Similarly, the second angle vector of point F and the second angle vector of point G can be obtained.

[0159] Then, point A in the overlapping area of the panoramic image is matched with points E, F and G in the close-up image, respectively. The specific matching method is to compare the first two bits of the first angle vector of point A with the second angle vectors of points E, F and G, respectively, to see if they are the same. For example, point A (15°, 30°, 120°) has two same bits compared with point E (15°, 30°), so it is recorded as 2. Assuming that one bit is the same compared with points F and G, it is recorded as 1. Finally, the Match_list of point E is [2, 1, 1]. The close-up target corresponding to the maximum matching index is taken as the matching target in the panoramic image. Finally, the matching target of the panoramic target point A is point E. All points are matched in turn.

[0160] In another possible implementation, after step S30, the method further includes:

[0161] Obtaining a first repeated person matching pair set determined by the quadrant vector;

[0162] Obtaining a second repeated person matching pair set determined by the angle vector;

[0163] Taking the union of the first repeated person matching pair set and the second repeated person matching pair set to determine the final repeated person.

[0164] It should be noted that after completing the similarity matching based on the quadrant vector and the angle vector, the first repeated person matching pair set determined by the quadrant vector matching or the second repeated person matching pair set determined by the angle vector matching is obtained accordingly.

[0165] In the specific embodiment, the first set of repeated person matching pairs comprises panorama target person and close-up target person pairs determined as the same person according to the high similarity of the sequence of the quadrant numbers in which the target persons are located in their respective coordinate systems, i.e. the relative position relationship. It can be understood that the quadrant matching mainly captures the discrete directional relationship between persons.

[0166] In the specific embodiment, the second set of repeated person matching pairs comprises panorama target person and close-up target person pairs determined as the same person according to the high similarity of the sequence of the angle values between the line connecting the target persons and the coordinate axis, i.e. the relative angle relationship. Exemplarily, the angle matching provides a continuous and refined spatial relationship measurement method.

[0167] It can be understood that due to the difference in perspective and detection bias, the matching pairs determined by the quadrant method and the angle method for the same target person may not completely coincide.

[0168] Finally, the union operation is performed on the first set of repeated person matching pairs and the second set of repeated person matching pairs to determine the final repeated person, wherein the final repeated person is the repeated person to be removed in the output picture. In the specific embodiment, this union means that as long as a certain target pair is determined as repeated by any one of the quadrant method or the angle method, it will be determined as a repeated person.

[0169] Optionally, this union strategy can ensure that as many real repeated targets as possible are captured, even if a certain method misses the judgment due to the limitation of a specific scene (such as the distribution of persons being exactly at the quadrant boundary), another method can successfully capture it. It can be understood that this mechanism of fusing two spatial relationship features combines the advantages of different geometric properties, forming a complementary deduplication scheme, which effectively improves the comprehensiveness and accuracy of the final output result.

[0170] As shown in FIG. 1, Figure 7 As shown in FIG. 1, Figure 7 The persons shown in FIG. 1 can be derived from a panorama wide-angle camera or a close-up fixed-focus camera, wherein the persons represented by reference number 200 and reference number 300 can be determined as repeated persons according to the quadrant matching method and the angle matching method of the present application, and only the angle of shooting is different, which belongs to the same person, so that in the final output picture, only the person in one of the images can be retained, which can be a panorama image or a close-up image.

[0171] In the embodiment, a fault-tolerant de-duplication method is provided by fusing the results of two independent judgment mechanisms, i.e., quadrant matching and angle matching. Specifically, the quadrant matching provides a macro spatial structure judgment through discrete orientation relationships, and the angle matching achieves a more fine spatial orientation verification through an included angle relationship. The dual-feature fusion strategy effectively widens the coverage of the matching conditions, so that even when one of the matching methods fails due to a change in the viewing angle or a detection error, the other matching method can still maintain good recognition ability. The decision method of parallel processing and then taking the union can achieve a more comprehensive and reliable person de-duplication effect in a complex multi-camera conference scene without excessively increasing the computational complexity.

[0172] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the multi-camera target de-duplication method of the present application. More forms of simple transformation based on the technical concept, such as interaction and combination of various embodiments, are within the protection scope of the present application.

[0173] The present application also provides a multi-camera target de-duplication device, which refers to Figure 8 The multi-camera target de-duplication device comprises:

[0174] An acquisition module 10 is configured to acquire a panoramic image and at least one close-up image.

[0175] A detection module 20 is configured to determine an overlapping area of the panoramic image and the close-up image, detect a first target person in the overlapping area in the panoramic image, and detect a second target person in the close-up image.

[0176] A matching module 30 is configured to perform similarity matching on a first spatial distribution relationship of the first target person in the overlapping area and a second spatial distribution relationship of the second target person in the close-up image, to obtain a similarity matching result.

[0177] A de-duplication module 40 is configured to determine a duplicate person according to the similarity matching result and remove the duplicate person from an output image, wherein the output image is a panoramic image or a close-up image in which the duplicate person is located.

[0178] Optionally, the matching module 30 is further configured to perform similarity matching on a first feature vector of the first target person in the overlapping area, which is generated based on a relative position relationship, and a second feature vector of the second target person in the close-up image, which is generated based on a relative position relationship, to obtain the similarity matching result.

[0179] Optionally, the matching module 30 is further configured to take the position of the first target person in the panoramic image as a first origin to establish a first coordinate system, calculate first relative position relationships of the remaining first target persons in the overlapping region relative to the first origin, and generate a first feature vector based on the first relative position relationships; wherein the first relative position relationships include quadrant relationships and included angle relationships, and the first feature vector includes a quadrant vector and an angle vector;

[0180] take the position of the second target person in the close-up image as a second origin to establish a second coordinate system, calculate second relative position relationships of the remaining second target persons in the close-up image relative to the second origin, and generate a second feature vector based on the second relative position relationships; wherein the second relative position relationships include quadrant relationships and included angle relationships, and the second feature vector includes a quadrant vector and an angle vector.

[0181] Optionally, the matching module 30 is further configured to determine first quadrant numbers of the remaining first target persons in the overlapping region in the first coordinate system, and generate first quadrant vectors of the first target persons according to the first quadrant numbers;

[0182] determine second quadrant numbers of the remaining second target persons in the close-up image in the second coordinate system, and generate second quadrant vectors of the second target persons according to the second quadrant numbers;

[0183] by traversing the first quadrant vector and the second quadrant vector, count the number of matching quadrants with the same quadrant number, and determine the similarity matching result based on the number of matching quadrants, wherein the more the number of matching quadrants is, the higher the similarity is.

[0184] Optionally, the matching module 30 is further configured to determine a first target included angle of the remaining first target persons in the overlapping region relative to the first origin in the first coordinate system; and generate a first angle vector of the first target person according to an angle value of the first target included angle;

[0185] determine a second target included angle of the remaining second target persons in the close-up image relative to the second origin in the second coordinate system; and generate a second angle vector of the second target person according to an angle value of the second target included angle;

[0186] traverse the first angle vector and the second angle vector, count the number of angles whose absolute difference of angle values at corresponding positions is less than a preset angle threshold, and determine the similarity matching result based on the number of angles, wherein the more the number of angles is, the higher the similarity is.

[0187] Optionally, the matching module 30 is further configured to obtain a first repeated person matching pair set determined through the quadrant vector;

[0188] obtaining a second set of repeated person matching pairs determined by the angle vector;

[0189] taking a union of the first set of repeated person matching pairs and the second set of repeated person matching pairs to determine a final repeated person.

[0190] The multi-camera target deduplication device provided in the present application adopts the multi-camera target deduplication method in the above embodiments, and can solve the technical problem of low accuracy of conference scene deduplication. Compared with the prior art, the multi-camera target deduplication device provided in the present application has the same beneficial effects as the multi-camera target deduplication method provided in the above embodiments, and other technical features in the multi-camera target deduplication device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0191] The present application provides a multi-camera target deduplication device, which comprises at least one processor and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the multi-camera target deduplication method in the first embodiment.

[0192] Reference will now be made to the drawings, and specific examples thereof will be illustrated. Figure 9 which shows a structural schematic diagram of a multi-camera target deduplication device suitable for implementing the embodiments of the present application. The multi-camera target deduplication device in the embodiments of the present application can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 9 The multi-camera target deduplication device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0193] As Figure 9As shown, the multi-camera target deduplication device can include a processing apparatus 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory 1002 or loaded from a storage apparatus 1003 into a random access memory 1004. Various programs and data required for the operation of the multi-camera target deduplication device are also stored in the random access memory 1004. The processing apparatus 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. An input / output interface 1006 is also connected to the bus. Generally, the following systems can be connected to the input / output interface 1006: input apparatuses 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output apparatuses 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage apparatus 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication apparatus 1009. The communication apparatus 1009 can allow the multi-camera target deduplication device to communicate with other devices wirelessly or by wire to exchange data. Although the multi-camera target deduplication device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0194] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication apparatus, or installed from the storage apparatus 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing apparatus 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are performed.

[0195] The multi-camera target deduplication device provided by the present application adopts the multi-camera target deduplication method in the above embodiments, and can solve the technical problem of low accuracy of conference scene deduplication. Compared with the prior art, the multi-camera target deduplication device provided by the present application has the same beneficial effects as the multi-camera target deduplication method provided by the above embodiments, and other technical features in the multi-camera target deduplication device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0196] It should be understood that various parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0197] The above description is merely that of a specific implementation of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, and all such changes or replacements should be encompassed within the scope of protection of the present application. Therefore, the scope of protection of the present application should be subject to the scope of protection of the claims.

[0198] The present application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., a computer program) for performing the multi-camera target deduplication method in the above embodiments.

[0199] The computer readable storage medium provided by the present application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), etc., or any suitable combination of the above.

[0200] The above computer readable storage medium can be contained in the multi-camera target deduplication device; or can exist separately and not be assembled into the multi-camera target deduplication device.

[0201] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the multi-camera target deduplication device, the multi-camera target deduplication device is caused to: acquire a panoramic image and at least one close-up image; determine an overlapping area of the panoramic image and the close-up image, detect a first target person in the overlapping area in the panoramic image, and detect a second target person in the close-up image; perform similarity matching on a first spatial distribution relationship of the first target person in the overlapping area and a second spatial distribution relationship of the second target person in the close-up image to obtain a similarity matching result; and determine a duplicate person according to the similarity matching result and remove the duplicate person from an output picture, wherein the output picture is the panoramic image or the close-up image in which the duplicate person is located.

[0202] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0203] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0204] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the names of the modules do not constitute a limitation on the modules themselves.

[0205] The computer readable storage medium provided in the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the multi-camera target deduplication method described above, and can solve the technical problem of low accuracy of conference scene deduplication. Compared with the prior art, the computer readable storage medium provided in the present application has the same beneficial effects as the multi-camera target deduplication method provided in the above embodiments, and will not be described here.

[0206] The present application also provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the multi-camera target deduplication method as described above.

[0207] The computer program product provided in the present application can solve the technical problem of low accuracy of conference scene deduplication. Compared with the prior art, the computer program product provided in the present application has the same beneficial effects as the multi-camera target deduplication method provided in the above embodiments, and will not be described here.

[0208] The above only describes some embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation, direct / indirect application in other related technical fields based on the technical concept of the present application, and the contents of the present application and the accompanying drawings are included in the patent protection scope of the present application.

Claims

1. A multi-site target deduplication method, characterized in that, The multi-camera target deduplication method comprises: acquiring a panoramic image and at least one close-up image; determining an overlapping area of the panoramic image and the close-up image, detecting a first target person in the overlapping area in the panoramic image, and detecting a second target person in the close-up image; performing similarity matching on a first spatial distribution relationship of the first target person in the overlapping area and a second spatial distribution relationship of the second target person in the close-up image to obtain a similarity matching result; The step of performing similarity matching on the first spatial distribution relationship of the first target person in the overlapping area and the second spatial distribution relationship of the second target person in the close-up image to obtain a similarity matching result comprises: taking the position of the first target person in the panoramic image as a first origin to establish a first coordinate system, calculating a first relative position relationship of the remaining first target persons in the overlapping area relative to the first origin, and generating a first feature vector based on the first relative position relationship; wherein the first relative position relationship comprises a quadrant relationship and an included angle relationship, and the first feature vector comprises a quadrant vector and an angle vector; taking the position of the second target person in the close-up image as a second origin to establish a second coordinate system, calculating a second relative position relationship of the remaining second target persons in the close-up image relative to the second origin, and generating a second feature vector based on the second relative position relationship; wherein the second relative position relationship comprises a quadrant relationship and an included angle relationship, and the second feature vector comprises a quadrant vector and an angle vector; performing similarity matching on the first feature vector of the first target person in the overlapping area generated based on the first relative position relationship and the second feature vector of the second target person in the close-up image generated based on the second relative position relationship to obtain a similarity matching result; According to the similarity matching result, determine the repeated person and remove the repeated person from the output picture, wherein the output picture is the panoramic image or the close-up image where the repeated person is located.

2. The method of claim 1, wherein, The step of generating a first feature vector based on the first relative position relationship comprises: determining the first quadrant number of the remaining first target persons in the overlapping area in the first coordinate system, and generating a first quadrant vector of the first target person according to the first quadrant number; The step of generating a second feature vector based on the second relative position relationship comprises: determining the second quadrant number of the remaining second target persons in the close-up image in the second coordinate system, and generating a second quadrant vector of the second target person according to the second quadrant number; The step of performing similarity matching on the first feature vector of the first target person in the overlapping area generated based on the relative position relationship and the second feature vector of the second target person in the close-up image generated based on the relative position relationship to obtain a similarity matching result comprises: The first quadrant vector and the second quadrant vector are traversed, the number of matching quadrants with the same quadrant number is counted, and a similarity matching result is determined based on the number of matching quadrants, wherein the more the number of matching quadrants is, the higher the similarity is.

3. The method of claim 1, wherein, The step of generating the first feature vector based on the first relative position relationship further includes: determining a first target included angle of the remaining first target persons in the overlapping area relative to the first origin in the first coordinate system, and generating a first angle vector of the first target person according to an angle value of the first target included angle; The step of generating the second feature vector based on the second relative position relationship further includes: determining a second target included angle of the remaining second target persons in the close-up image relative to the second origin in the second coordinate system, and generating a second angle vector of the second target person according to an angle value of the second target included angle; The step of performing similarity matching between the first feature vector of the first target person in the overlapping area based on the relative position relationship and the second feature vector of the second target person in the close-up image based on the relative position relationship to obtain a similarity matching result further includes: The first angle vector and the second angle vector are traversed, the number of angles with an absolute difference of corresponding position angle values less than a preset angle threshold is counted, and a similarity matching result is determined based on the number of angles, wherein the more the number of angles is, the higher the similarity is.

4. The method of claim 1, wherein, After the step of performing similarity matching between the first spatial distribution relationship of the first target person in the overlapping area and the second spatial distribution relationship of the second target person in the close-up image to obtain a similarity matching result, the method further includes: obtaining a first repeated person matching pair set determined by the quadrant vector; obtaining a second repeated person matching pair set determined by the angle vector; performing a union of the first repeated person matching pair set and the second repeated person matching pair set to determine a final repeated person.

5. A multi-silo target deduplication apparatus, characterized by, The multi-camera target deduplication device includes: an acquisition module configured to acquire a panoramic image and at least one close-up image; a detection module configured to determine an overlapping area of the panoramic image and the close-up image, detect first target persons in the overlapping area in the panoramic image, and detect second target persons in the close-up image; a matching module configured to perform similarity matching between a first spatial distribution relationship of the first target persons in the overlapping area and a second spatial distribution relationship of the second target persons in the close-up image to obtain a similarity matching result; the matching module is configured to take a position of the first target person in the panoramic image as a first origin to establish a first coordinate system, calculate a first relative position relationship of the remaining first target persons relative to the first origin, and generate a first feature vector based on the first relative position relationship; wherein the first relative position relationship includes a quadrant relationship and an included angle relationship, and the first feature vector includes a quadrant vector and an angle vector; the matching module is configured to take a position of the first target person in the panoramic image as a first origin to establish a first coordinate system, calculate a first relative position relationship of the remaining first target persons relative to the first origin, and generate a first feature vector based on the first relative position relationship; wherein the first relative position relationship includes a quadrant relationship and an included angle relationship, and the first feature vector includes a quadrant vector and an angle vector; The position of the second target person in the close-up image is taken as a second origin to establish a second coordinate system, second relative position relationships of the rest of the second target persons in the close-up image relative to the second origin are calculated, and a second feature vector is generated based on the second relative position relationships; wherein the second relative position relationships include quadrant relationships and included angle relationships, and the second feature vector includes a quadrant vector and an angle vector; The first feature vector of the first target person in the overlapping area generated based on the first relative position relationships is subjected to similarity matching with the second feature vector of the second target person in the close-up image generated based on the second relative position relationships, to obtain a similarity matching result; The de-duplication module is configured to determine a duplicate person according to the similarity matching result and remove the duplicate person from an output picture, wherein the output picture is a panoramic image or a close-up image in which the duplicate person is located.

6. A multi-silo target deduplication device, comprising: The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the multi-camera target de-duplication method according to any one of claims 1 to 4.

7. A storage medium, characterized by The storage medium is a computer-readable storage medium, and the storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the multi-camera target de-duplication method according to any one of claims 1 to 4.

8. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by the processor to implement the steps of the multi-camera target de-duplication method according to any one of claims 1 to 4.

Images