Multi-camera target positioning and identification method and system based on edge calculation

Abstract

The invention discloses a multi-camera target positioning and identification method and system based on edge calculation. The method comprises the following steps: acquiring video or image data acquired by cameras; for each camera, utilizing a four-point positioning method to confirm a conversion matrix from a pixel coordinate to a world coordinate in a picture of each camera in an area in a visual angle of the camera; according to the collected video or image data, through a YOLO target recognition model and a YOLO target recognition model, obtaining human body and face feature data in a target picture; and converting the pixel coordinates of the identified target in the camera picture into the world coordinates by using the conversion matrix from the pixel coordinates of the camera to the world coordinates. By adopting the technical scheme of the invention, the accuracy and robustness of personnel identification and positioning are improved in a large-scale complex scene, and the flexibility and expandability of the system are improved by utilizing an edge computing architecture.

Description

A Multi-Camera Target Localization and Recognition Method and System Based on Edge Computing Technical Field

[0001] This invention belongs to the field of video surveillance technology, specifically relating to a multi-camera target localization and recognition method and system based on edge computing. Background Technology

[0002] In the development of smart cities and smart security applications, personnel target recognition and localization based on video surveillance plays a crucial role. Traditional methods mainly rely on video footage from a single camera, which can basically achieve personnel detection, recognition, and location. However, due to the limited field of view of a single camera and its susceptibility to noise interference in the monitored footage, traditional single-camera-based personnel detection, recognition, and localization methods struggle to handle complex scenes with large areas and multiple obstructions, resulting in low recognition rates and poor positioning accuracy, making them unsuitable for deployment in application scenarios such as large public places.

[0003] To achieve personnel localization based on camera video footage, it is necessary to determine the transformation relationship between the pixel coordinates of the personnel in the camera image and world coordinates. Traditional methods require precise camera calibration and complex geometric calculations to determine this transformation relationship, significantly increasing the system's complexity.

[0004] Existing methods for personnel detection, recognition, and localization based on camera footage require image feature extraction operations such as optical flow algorithms and convolution on the collected video footage, which incurs significant computational overhead. In scenarios where multiple cameras are used for personnel recognition and localization, the need to process video footage from multiple cameras simultaneously, coupled with the limited number of cameras that a single computing device can connect to, makes it difficult for a single computing device to meet the computational processing and camera connectivity requirements for multi-camera target recognition and localization in complex scenarios. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a multi-camera target localization and recognition method and system based on edge computing. This method improves the accuracy and robustness of personnel identification and localization in large-scale complex scenarios, and enhances the system's flexibility and scalability by utilizing an edge computing architecture.

[0006] To achieve the above objectives, the present invention provides the following solution: a multi-camera target recognition and localization method based on edge computing, comprising: acquiring video or image data collected by cameras; determining the transformation matrix from pixel coordinates to world coordinates in the field of view of each camera; obtaining human body and face feature data in the target image through a YOLO target recognition model and a re-recognition model based on the acquired video or image data; and converting the pixel coordinates of the target identified in the camera image into world coordinates using the transformation matrix from camera pixel coordinates to world coordinates.

[0007] As a preferred approach, a pre-trained YOLO target recognition model is used to identify targets in the camera image. Subsequently, the identified target images are normalized, and a re-identification model based on the OSNet structure, implemented in the torchreid framework, is used to extract feature data such as human bodies and faces from the target images.

[0008] Preferably, coordinates from different cameras with a Euclidean distance difference of less than a predetermined range are assigned to the same target to obtain a valid coordinate dataset for each target; the average of the coordinates in the coordinate dataset is then used as the world coordinates of the target in the current frame.

[0009] As a preferred method, the feature data of the located target is compared with a pre-established target feature database. Based on the degree of similarity, a confidence level for target recognition is assigned, and the target identity recognition result in the current camera image is determined accordingly. For the identity recognition results of each camera for the same located target, a voting method is used to obtain the final identity recognition result of the target.

[0010] As a preferred option, establishing a target feature database specifically includes: acquiring multiple images of the same target under different viewpoints, lighting conditions, and poses; extracting feature vectors from the images; averaging several feature vectors obtained for the same target to obtain an average feature vector, which is then used as the feature vector corresponding to the target and stored in the feature database.

[0011] As a preferred method, for each camera, within its field of view, the transformation matrix from pixel coordinates to world coordinates in each camera's image is determined using the four-point positioning method.

[0012] This invention also provides a multi-camera target recognition and localization system based on edge computing. Multiple cameras, together with an edge computing device, form a device group to identify and locate targets within a spatial area of ​​an application scenario. By deploying multiple such device groups, the target recognition and localization information calculated by the edge computing devices in each device group is aggregated to a user-specified local host or network address, completing target recognition and localization across the entire spatial area of ​​the application scenario. The edge computing device includes: a memory, an external device connection port, and a processor. The memory stores computer programs; the external device connection port connects to multiple external cameras to collect image data from each camera; the processor has graphics processing acceleration capabilities and can execute the computer program, enabling the edge computing device to perform the edge computing-based multi-camera target recognition and localization method.

[0013] Preferably, multiple cameras are connected to the edge computing device through the external device connection port of the edge computing device, and the camera video streams are transmitted to the edge computing device in real time; the edge computing device executes a multi-camera target recognition and localization method based on edge computing on the received video streams from each camera.

[0014] Compared with existing technologies, the beneficial effects of this invention are as follows: 1. This invention spatially fuses the target coordinate data of multi-camera target localization by averaging, and introduces a voting mechanism for the identified target identity. Compared with traditional single-camera target recognition and localization methods, the method proposed in this invention significantly improves the accuracy and robustness of target localization and identity recognition in complex scenarios; 2. This invention solves the problem of excessive computing power requirements and heavy camera connection pressure caused by using a single computing device for centralized processing in traditional methods by delegating the calculation process of multi-camera target recognition and localization to multiple computing devices described in the invention; 3. This invention can flexibly increase or decrease the device group consisting of the computing device described in the invention and multiple cameras according to the actual application scenario, and deploy it in various sub-space areas of the application scenario, which has high flexibility and scalability. Attached Figure Description

[0015] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 is a flowchart of the target recognition and localization method based on multiple cameras according to an embodiment of the present invention; Figure 2 is a schematic diagram of the system architecture and deployment of the multi-camera target recognition and localization system based on edge computing according to an embodiment of the present invention; Figure 3 is a schematic diagram of the basic structure of the YOLO target recognition algorithm used in an embodiment of the present invention; Figure 4 is a schematic diagram of the re-recognition model based on the OSNet structure used in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] As shown in Figure 1, Example 1 of this invention provides a target localization and recognition method based on multi-camera image information, including: using a target detection model to detect targets in video frames captured by each camera and obtaining the pixel position of the target in the image; then using a re-identification model to extract the feature information of the target; next, converting the pixel coordinates of the target into physical world coordinates; finally, the system fuses the spatial coordinates of the same target from multiple cameras and determines the target identity based on the similarity of features.

[0020] This embodiment performs target detection based on the YOLO algorithm on real-time video streams captured by multiple cameras to obtain the pixel positions of the targets in the images. It then uses an OSNet-based re-identification model to extract high-dimensional feature vectors of the targets. Finally, combining this with a pre-calibrated homography matrix, the pixel coordinates are converted into physical world coordinates. Finally, the system performs spatial fusion on the spatial coordinates of the same target from multiple cameras by averaging the results, and uses a voting method based on the similarity of the feature vectors to confirm the target's identity.

[0021] This embodiment is based on a purely visual approach, without relying on additional equipment such as LiDAR. It can achieve high-precision positioning using only ordinary surveillance cameras, making it cost-effective. This embodiment utilizes the complementarity of multiple camera perspectives; when a target is occluded from one perspective, the system can still continuously track and locate it through other perspectives, effectively solving the problem of target loss in complex scenarios.

[0022] The multi-camera target localization and recognition method in this implementation example includes the following steps: Step S1: Establish coordinate mapping relationship using the four-point positioning method and establish a target feature database; Step S2: Acquire video or image data collected by the cameras, perform target detection using a pre-trained target recognition model, and extract features using a re-recognition model; Step S3: Multi-camera coordinate fusion and target localization.

[0023] Step S4: Identity recognition based on feature comparison and voting method.

[0024] Specifically, step S1 includes the following steps: S1-1: During the system initialization phase, for each camera... The field of view is determined by selecting four known coordinate markers in the real physical environment. Let the coordinates of these four points in the world coordinate system be... The corresponding coordinates in the camera image pixel coordinate system are .

[0025] This invention utilizes the principle of perspective transformation to establish a mapping relationship; the transformation relationship between pixel coordinates and world coordinates can be achieved through a homography matrix. express: The matrix can be obtained by using the coordinates of four corresponding points and a direct linear transformation algorithm. The parameters in [the dataset]. For any target pixel center point detected in real time. The corresponding physical world coordinates can then be calculated using matrix multiplication and the aforementioned relationships. .

[0026] S1-2: Establish a feature database of known targets. First, for each target person to be registered, acquire multiple clear portrait images under different camera angles, lighting conditions, and body poses. Each acquired image undergoes the same preprocessing as in step S2-2 (normalizing the image size through deformation). After the pixel number is calculated, it is input into the OSNet-based re-identification model, the structure of which is shown in Figure 4. The model outputs the original feature vector corresponding to each image. For the same target, all feature vectors obtained from the acquired images. Calculate the arithmetic mean to obtain the representative average eigenvector of the target. : Subsequently, regarding L2 normalization is performed, and the normalized vector is associated with the target's identity ID and stored in the feature database for subsequent identity matching.

[0027] Step S2 includes the following steps: S2-1: After receiving the video stream, the edge computing device uses the YOLO target detection model to locate and crop the target human image. The structure of the YOLO object detection model is shown in Figure 3.

[0028] S2-2: Will Transform to a uniform size (e.g.) The pixels are normalized and preprocessed, and then input into the OSNet-based re-identification network. The structure of the re-identification model is shown in Figure 4.

[0029] The output layer of the re-identification network maps the image to a... Generate high-dimensional feature vectors of the target in a 3D feature space. The feature vector is then L2 normalized to obtain the final query vector. : ;in .

[0030] Step S3 includes the following steps: S3-1: Collect data from... The system calculates the target detection results from multiple cameras at the same time stamp and uses the transformation relationship between pixel coordinates and world coordinates in S1-1 to convert the target's pixel coordinates to world coordinates. Let the data originate from the camera... and cameras The target world coordinates after transformation are as follows: and .

[0031] S3-2: Calculate the Euclidean distance between two points. : Set distance threshold (For example, 1.5 meters). If If the two detection results indicate that they belong to the same target in physical space, the detected world coordinates will be added to the target's coordinate set. .

[0032] S3-3: Set of coordinates for each target The final positioning coordinates are obtained by averaging all world coordinates within the range. : ;in The set contains the number of coordinates, corresponding to the number of cameras that successfully detected the target.

[0033] Step S4 includes the following steps: using the located target The corresponding normalized feature vector is used as the query vector. ;Will With the feature database, all registered identities Feature vector set Compare them.

[0034] As described in process S1-2, the vectors in the feature database All of these are pre-extracted through the aforementioned re-identification model and then normalized using L2.

[0035] S4-1: Calculate the query vector Compared with all feature vectors in the database cosine similarity .because and All have been L2 normalized, i.e., the modulus is 1, and the similarity is... The solution can be simplified to the dot product of two vectors: ; here This is the identification confidence level, and its value ranges from [0, 1].

[0036] S4-2: System Finds and Queries Vectors The most similar database vectors : If the highest similarity Greater than the preset identity verification threshold (For example If the initial identification result of the target is determined as... Corresponding identity Otherwise, the target is marked as "unknown".

[0037] S4-3: The system collects all Multiple cameras, all focused on the same target. Set of preliminary identity recognition results within the same time window The final identity of the target is determined by majority voting. : That is, select the identity that appears most frequently among all valid recognition results as the final output.

[0038] Example 2, as shown in Figure 2, also provides a multi-camera target recognition and localization system based on edge computing, comprising: an edge computing device, which serves as an edge computing node of the system, and includes a memory, an external device connection port, and a processor. The processor should have graphics processing acceleration capabilities, such as an embedded processor integrating a CUDA core, to perform the computational processes required by the target localization and recognition method; the memory is used to store computer programs, which, when executed, can complete all or part of the functions of the target recognition and localization method, or control the target recognition and localization system; the external device connection interface is used to connect multiple cameras to the computing device, and its form can be an Ethernet port supporting the RTSP protocol.

[0039] In practical applications, several cameras are connected to a single edge computing device to form a device group responsible for monitoring a specific sub-area. Multiple such device groups are deployed depending on the scene area. Each device group independently performs image acquisition, target detection, feature extraction, and coordinate transformation within its assigned spatial area. After processing, each edge computing device sends the processed structured data, including target identification results and target world coordinates, to a designated local host or network address via the network. The designated local host or the host corresponding to the network address collects the structured data sent by each edge computing device to complete the collection of target location and identification results throughout the entire application scene.

[0040] By constructing a multi-camera target localization and recognition system based on edge computing, the connections of multiple cameras are distributed to various edge computing devices, and computationally intensive image processing tasks are offloaded to edge computing devices at the edge, only sending back structured data. This solves the problems of limited device connectivity and excessively high computing power requirements encountered in the traditional centralized processing method using a single computing device.

[0041] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A multi-camera target recognition and localization method based on edge computing, characterized in that, include: Acquire video or image data captured by the camera; Within the field of view of each camera, confirm the transformation matrix from pixel coordinates to world coordinates in each camera's image; based on the collected video or image data, obtain human body and facial feature data in the target image through the YOLO target recognition model and re-recognition model; use the transformation matrix from camera pixel coordinates to world coordinates to convert the pixel coordinates of the target recognized in the camera image into world coordinates.

2. The multi-camera target recognition and localization method based on edge computing as described in claim 1, characterized in that, Using a pre-trained YOLO target recognition model, targets in the camera image are identified. Then, the identified target images are normalized, and a re-identification model based on the OSNet structure, implemented in the torchreid framework, is used to extract feature data such as human body and face from the target image.

3. The multi-camera target recognition and localization method based on edge computing as described in claim 2, characterized in that, Coordinates from different cameras with a difference in Euclidean distance less than a predetermined range are assigned to the same target to obtain a valid coordinate dataset for each target. The average value of the coordinates in the coordinate dataset is then used as the world coordinates of the target in the current frame.

4. The multi-camera target recognition and localization method based on edge computing as described in claim 3, characterized in that, The feature data of the located target is compared with a pre-established target feature database. Based on the degree of similarity, a confidence level for target recognition is assigned, which determines the target identity recognition result in the current camera image. For the identification results of the same located target from each camera, a voting method is used to obtain the final identification result of the target.

5. The multi-camera target recognition and localization method based on edge computing as described in claim 4, characterized in that, Establishing a target feature database specifically includes: acquiring multiple images of the same target under different viewpoints, lighting conditions, and poses; extracting feature vectors from the images; averaging several feature vectors obtained for the same target to obtain an average feature vector, which is then used as the feature vector corresponding to the target and stored in the feature database.

6. The multi-camera target recognition and localization method based on edge computing as described in claim 5, characterized in that, For each camera, within its field of view, the transformation matrix from pixel coordinates to world coordinates in each camera's image is determined using the four-point positioning method.

7. A multi-camera target recognition and localization system based on edge computing, characterized in that, Multiple cameras, together with an edge computing device, form a device group to identify and locate targets within a spatial area of ​​an application scenario. By deploying multiple such device groups, the target identification and location information calculated by the edge computing device in each device group is aggregated to a user-specified local host or network address, thereby completing target identification and location for the entire spatial area of ​​the application scenario. The edge computing device includes: a memory, an external device connection port, and a processor. The memory stores computer programs. The external device connection port connects to multiple external cameras to collect image data from each camera. The processor should have graphics processing acceleration capabilities and be able to execute the computer program to enable the edge computing device to perform all the methods described in claims 1-6.

8. The multi-camera target recognition and localization system based on edge computing according to claim 7, characterized in that, Multiple cameras are connected to the edge computing device via the external device connection port of the edge computing device, and the camera video streams are transmitted to the edge computing device in real time. The edge computing device performs the calculations required by the method described in claims 1-6 on the received video streams from each camera.

Images