Multi-panoramic-camera target tracking method and device, computer equipment and storage medium

Abstract

The invention relates to a multi-panoramic-camera target tracking method and device, computer equipment and a storage medium. The multiple panorama cameras comprise a main panorama camera and a plurality of auxiliary panorama cameras, the main panorama camera is installed on an amusement vehicle, and the method comprises the steps that first feature information sent by the main panorama camera is received; the first feature information is feature information, shot by the main panorama camera, of a target person riding the amusement vehicle; performing pedestrian re-identification on the photographed figures according to the first feature information, and determining matched figures matched with the first feature information of the target figure in the figures photographed by the auxiliary panorama camera; receiving a first clustering center determined after clustering of first feature information sent by the main panorama camera, determining second feature information corresponding to the matched person, and clustering the second feature information to determine a second clustering center; and according to the first clustering center and the second clustering center, performing target tracking on the target person shot in the auxiliary panorama camera.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a multi-panoramic camera target tracking method, device, computer equipment, and storage medium. Background Art

[0002] With the development of mobile portable filming equipment, for entertainment and leisure scenes such as karts and amusement parks, the range of tourists' positions is large and the angles of human bodies vary greatly. If tourists want to capture images of wonderful moments during the game, using a single mobile portable filming device may not be able to fully record the wonderful moments. In this regard, entertainment and leisure scenes such as karts and amusement parks can be equipped with multiple mobile portable filming devices, which can be used to identify tourists and record various wonderful moments of tourists.

[0003] However, when multiple mobile portable shooting devices are identifying tourists, the tourists may be blocked in the entertainment scene, resulting in the body not being fully visible. Therefore, how to accurately identify the target tourists and export video data that matches the target tourists is a problem that needs to be solved. Summary of the invention

[0004] Based on this, it is necessary to provide a multi-panoramic camera target tracking method, device, computer equipment, and storage medium to address the above technical problems.

[0005] In a first aspect, the present disclosure provides a multi-panoramic camera target tracking method. The multi-panoramic camera includes: a main panoramic camera and a plurality of secondary panoramic cameras, the main panoramic camera is installed on an amusement vehicle, and the method includes:

[0006] receiving first characteristic information sent by the main panoramic camera; the first characteristic information is characteristic information of a target person riding in the amusement vehicle photographed by the main panoramic camera;

[0007] Re-identify the captured person according to the first feature information, and determine a matching person among the persons captured by the secondary panoramic camera that matches the first feature information of the target person;

[0008] receiving a first cluster center determined after clustering the first feature information sent by the main panoramic camera, and determining second feature information corresponding to the matching person and clustering the second feature information to determine a second cluster center;

[0009] According to the first cluster center and the second cluster center, target tracking is performed on the target person captured by the secondary panoramic camera.

[0010] In one embodiment, tracking the target person captured by the secondary panoramic camera according to the first cluster center and the second cluster center includes:

[0011] Calculating the similarity between the first cluster center and the second cluster center;

[0012] According to the similarity and a preset similarity threshold, the second cluster centers are screened to determine a target cluster center among the second cluster centers;

[0013] Target feature information matching the target cluster center is determined, a target person photographed by the secondary panoramic camera is determined according to the target feature information, and target tracking is performed on the target person.

[0014] In one embodiment, the first characteristic information is obtained by:

[0015] Determine a plurality of panoramic images taken by the main panoramic camera, and extract character feature information of characters contained in each panoramic image;

[0016] According to the character feature information of the characters contained in each frame of the panoramic image and the time of each frame of the panoramic image, a character time relationship matrix is ​​constructed;

[0017] Calculating similarity scores between character feature information at different times in the character time relationship matrix;

[0018] The first characteristic information of the target person contained in the person time relationship matrix is ​​determined according to the similarity score and a predetermined association algorithm.

[0019] In one embodiment, tracking the target person captured by the secondary panoramic camera according to the first cluster center and the second cluster center includes:

[0020] receiving first time information corresponding to the first cluster center in each frame of a first panoramic image sent by the main panoramic camera, where the first panoramic image is an image captured by the main panoramic camera;

[0021] Determining second time information corresponding to the second cluster center in each frame of a second panoramic image, where the second panoramic image is an image captured by the secondary panoramic camera;

[0022] constructing a first association matrix according to the first cluster center and the first time information, and constructing a second association matrix according to the second cluster center and the second information;

[0023] Calculating the similarity between the first incidence matrix and the second incidence matrix, screening the second incidence matrix according to the similarity, and determining a target element in the second incidence matrix;

[0024] According to the target element, a target person photographed by the secondary panoramic camera is determined, and target tracking is performed on the target person.

[0025] In one embodiment, after tracking the target person, the method further includes:

[0026] Determine a plurality of panoramic images captured by the secondary panoramic camera and containing the target person;

[0027] The multi-frame panoramic image containing the target person is sent to the main panoramic camera to instruct the main panoramic camera to perform video editing based on the multi-frame panoramic image and the panoramic image containing the target person taken by the main panoramic camera to obtain a video containing the target person.

[0028] In one of the embodiments, before receiving the first feature information sent by the main panoramic camera, the method further includes:

[0029] A connection request is sent to the main panoramic camera, where the connection request is used to instruct the main panoramic camera to establish a connection with the secondary panoramic camera.

[0030] In a second aspect, the present disclosure further provides a multi-panoramic camera target tracking device. The multi-panoramic camera includes: a main panoramic camera and a plurality of sub-panoramic cameras, the main panoramic camera is installed on an amusement vehicle, and the device includes:

[0031] An information receiving module, used for receiving first characteristic information sent by the main panoramic camera; the first characteristic information is characteristic information of a target person riding in the amusement vehicle photographed by the main panoramic camera;

[0032] a matching person determination module, configured to perform person re-identification on the photographed person according to the first feature information, and determine a matching person among the persons photographed by the secondary panoramic camera that matches the first feature information of the target person;

[0033] a center point determination module, configured to receive a first cluster center determined after clustering the first feature information and sent by the main panoramic camera, and to determine second feature information corresponding to the matching person and cluster the second feature information to determine a second cluster center;

[0034] A target tracking module is used to track the target person captured by the secondary panoramic camera according to the first cluster center and the second cluster center.

[0035] In a third aspect, the present disclosure further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in any of the above method embodiments are implemented.

[0036] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0037] In a fifth aspect, the present disclosure further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0038] In each of the above embodiments, the first feature information sent by the main panoramic camera is received. The first feature information is the feature information of the target person riding the amusement vehicle photographed by the main panoramic camera. According to the first feature information, the photographed person is re-identified, and the matching person among the persons photographed by the secondary panoramic camera that matches the first feature information of the target person is determined. The pedestrian re-identification can accurately identify the person through the first feature information to obtain the matching person, thereby ensuring the accuracy of subsequent target tracking. In addition, since pedestrian re-identification may also have the case of misidentification, if misidentification occurs, the wrong person will be followed in the subsequent tracking process, resulting in the inability to track the target person. Therefore, in order to avoid this situation, the second feature information of the matching person can also be determined, the second feature information is clustered, and the second cluster center is obtained. According to the first cluster center and the second cluster center, the matching person can be further screened, thereby determining the target person, and the target person photographed by the secondary panoramic camera is tracked, and the wonderful moments of the target person can be accurately recorded. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1A schematic diagram of an application environment of a multi-panoramic camera target tracking method in an embodiment;

[0041] Figure 2 A schematic diagram of a flow chart of a multi-panoramic camera target tracking method in one embodiment;

[0042] Figure 3 A schematic diagram of a flow chart of step S208 in an embodiment;

[0043] Figure 4 A schematic diagram of a process for obtaining first characteristic information in an embodiment;

[0044] Figure 5 Another schematic diagram of a flow chart of step S208 in one embodiment;

[0045] Figure 6 A schematic diagram of the process after step S208 in one embodiment

[0046] Figure 7 It is a schematic block diagram of the structure of a multi-panoramic camera target tracking device in one embodiment;

[0047] Figure 8 Schematic diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0049] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0050] In this article, the term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship.

[0051] The present disclosure provides a multi-panoramic camera target tracking method, which can be applied to Figure 1 In the application environment shown. Among them, the main panoramic camera 102 communicates with multiple sub-panoramic cameras 104. The main panoramic camera 102 is usually installed on an amusement vehicle. The sub-panoramic camera 104 receives the first feature information sent by the main panoramic camera 102. The first feature information is the feature information of the target person riding the amusement vehicle photographed by the main panoramic camera 102. The sub-panoramic camera 104 performs pedestrian re-identification on the photographed person according to the first feature information, and determines the matching person that matches the first feature information of the target person among the people photographed by the sub-panoramic camera 104. The sub-panoramic camera 104 receives the first cluster center obtained by clustering the first feature information sent by the main panoramic camera 102, and determines the second feature information corresponding to the matching person and clusters the second feature information to obtain the second cluster center. The sub-panoramic camera 104 tracks the target person photographed by the sub-panoramic camera according to the first cluster center and the second cluster center.

[0052] In one embodiment, Figure 2 As shown, a multi-panoramic camera target tracking method is provided, and the method is applied to Figure 1 Taking the secondary panoramic camera 104 in the figure as an example, the multi-panoramic camera may include: a main panoramic camera and a secondary panoramic camera. The main panoramic camera is usually installed on an amusement vehicle, mainly to shoot the scenes of tourists or users riding on the amusement vehicle and driving the amusement vehicle. The secondary panoramic camera can be installed in various locations in entertainment and leisure venues, such as the boundary of a racing track, on other amusement vehicles, and so on. It should be noted that in some embodiments of the present disclosure, the panoramic camera installed on the vehicle ridden by the target person may be the main panoramic camera, and the panoramic cameras installed on the remaining amusement vehicles may be secondary panoramic cameras. Depending on the amusement vehicle ridden by the target person, the main panoramic camera and the secondary panoramic camera will change accordingly, and the amusement vehicles may include: karts, sightseeing cars, cable cars, and so on. The method comprises the following steps:

[0053] S202, receiving first feature information sent by the main panoramic camera; the first feature information is feature information of a target person riding in the amusement vehicle photographed by the main panoramic camera.

[0054] The characteristic information of the target person refers to various attributes and features used to describe and identify the target person, and may include, for example, body features, appearance features, clothing features, and the like.

[0055] Specifically, since the main panoramic camera is installed on the amusement vehicle, when a person rides the amusement vehicle, the panoramic camera can identify the person riding on the amusement vehicle, and the person riding on the amusement vehicle can be the target person. When the main panoramic camera identifies the target person, the characteristic information of the target person can be identified, thereby obtaining the first characteristic information. In addition, the main panoramic camera can also detect whether there is a secondary panoramic camera within a certain range. If there is a secondary panoramic camera, the first characteristic information can be sent to the secondary panoramic camera. The secondary panoramic camera can receive the first characteristic information.

[0056] S204: Re-identify the photographed person according to the first feature information, and determine a matching person among the persons photographed by the secondary panoramic camera that matches the first feature information of the target person.

[0057] Person Re-identification (ReID) is an important research task in the field of computer vision, which is mainly used to identify the same person between different cameras. The goal of ReID is to enable the same individual to be recognized when photographed from different perspectives, different lighting conditions, and different backgrounds.

[0058] Specifically, the secondary panoramic camera may use the first feature information to perform pedestrian re-identification on the panoramic image captured by it, thereby determining a matching person in the panoramic image captured by it that matches the first feature.

[0059] S206, receiving a first cluster center determined after clustering the first feature information sent by the main panoramic camera, and determining second feature information corresponding to the matching person and clustering the second feature information to determine a second cluster center.

[0060] Among them, cluster center is a key concept in cluster analysis. It refers to the central position or representative point corresponding to each cluster (or cluster) in the clustering algorithm. The calculation of cluster center is usually based on the characteristics of all sample points in the cluster and is used to represent the core characteristics of the cluster.

[0061] Specifically, after obtaining the first feature information, the main panoramic camera can cluster the first feature information to determine the first cluster center. Clustering algorithms such as K-means clustering, hierarchical clustering, and DBSCAN (density-based clustering) can be used for clustering. Similarly, after the secondary panoramic camera determines the matching person, the secondary panoramic camera can identify the second feature information of the matching person. Then the second feature information is clustered to obtain the second cluster center.

[0062] S208: Track the target person captured by the secondary panoramic camera according to the first cluster center and the second cluster center.

[0063] Among them, object tracking is an important task in the field of computer vision and image processing, which aims to continuously detect and track the movement of specific objects from video sequences or continuous frames.

[0064] Specifically, a second cluster center that matches the first cluster center can be determined from the first cluster center. Then, the target person is determined using the second cluster center that matches the first cluster center. After determining the target person, the secondary panoramic camera can track the target person within the recognition range.

[0065] In the above multi-panoramic camera target tracking method, the first feature information sent by the main panoramic camera is received. The first feature information is the feature information of the target person riding the amusement vehicle photographed by the main panoramic camera. According to the first feature information, the photographed person is re-identified, and the matching person that matches the first feature information of the target person among the persons photographed by the secondary panoramic camera is determined. The pedestrian re-identification can accurately identify the person through the first feature information to obtain the matching person, thereby ensuring the accuracy of subsequent target tracking. In addition, since pedestrian re-identification may also have the case of misidentification, if misidentification occurs, the wrong person will be followed in the subsequent tracking process, resulting in the inability to track the target person. Therefore, in order to avoid this situation, the second feature information of the matching person can also be determined, the second feature information is clustered, and the second cluster center is obtained. According to the first cluster center and the second cluster center, the matching person can be further screened, thereby determining the target person, and the target person photographed by the secondary panoramic camera is tracked, and the wonderful moments of the target person can be accurately recorded.

[0066] In one embodiment, Figure 3As shown, the target person captured by the secondary panoramic camera is tracked according to the first cluster center and the second cluster center, including:

[0067] S302: Calculate the similarity between the first cluster center and the second cluster center.

[0068] Specifically, the Euclidean distance between the first cluster center and the second cluster center can be calculated, and the similarity can be determined based on the Euclidean distance. The cosine similarity between the first cluster center and the second cluster center can also be calculated, and the similarity can be determined based on the cosine similarity. In addition, if the first cluster center and the second cluster center are binary features, the Jaccard similarity coefficient can be used to calculate the similarity between the first cluster center and the second cluster center.

[0069] S304: Screen the second cluster centers according to the similarity and a preset similarity threshold, and determine a target cluster center among the second cluster centers.

[0070] Specifically, the similarity threshold can be used to screen the similarity, and the similarity greater than the similarity threshold can be determined. Then the second cluster center corresponding to the similarity greater than the similarity threshold can be determined. The second center point is the target cluster center after screening.

[0071] S306, determining target feature information matching the target cluster center, determining a target person photographed by the secondary panoramic camera according to the target feature information, and performing target tracking on the target person.

[0072] Specifically, after determining the target cluster center, the second feature information matching the target cluster center can be found, and the second feature information can be determined as the target feature information. After finding the person who generated the target feature information, the person can be the target person. After determining the target person, the target person can be tracked.

[0073] In this embodiment, by calculating the similarity and using the similarity threshold to determine the target cluster center, the target person is determined, which can more accurately determine the target person and improve the accuracy of target tracking.

[0074] In one embodiment, Figure 4 As shown, the first characteristic information includes being obtained in the following manner:

[0075] S402, determining a plurality of panoramic images taken by the main panoramic camera, and extracting character feature information of characters contained in each panoramic image.

[0076] Specifically, during the shooting process, the main panoramic camera can first identify the person in each panoramic image frame, and then determine the person's characteristic information. The person in the panoramic image can be identified by image segmentation or by a neural network model such as YOLO (You Only Look Once).

[0077] S404: construct a character time relationship matrix according to the character feature information of the characters contained in each frame of the panoramic image and the time of each frame of the panoramic image.

[0078] Specifically, a correlation matrix may be constructed based on the character feature information of each frame of image and the time information of each frame of panoramic image. The correlation matrix may include: the character feature information contained in each time information.

[0079] S406, calculating similarity scores between character feature information at different times in the character time relationship matrix.

[0080] Specifically, for targets at different times, similarity scores between character feature information at different times may be calculated.

[0081] S408, determining first feature information of the target person contained in the person time relationship matrix according to the similarity score and a predetermined association algorithm.

[0082] The association algorithms may include: Hungarian algorithm, Kalman filtering, multiple hypothesis tracking (MHT) and other algorithms.

[0083] Specifically, the relevant character feature information can be determined based on the similarity score and the association algorithm, and the same character feature information between different panoramic images is determined based on the relevant character feature information. The first feature information is determined based on the character feature information that appears in each panoramic image frame.

[0084] In some exemplary embodiments, taking two panoramic images as an example, two panoramic images t and t+1 as an example, an association matrix A can be created, the size of the matrix is ​​M×N, where M is the first feature information in t, and N is the first feature information in time t+1. The element A[i][j] of the matrix represents the similarity score between the first feature information of the i-th target in t and the first feature information of the j-th target in t+1. Then, the association matrix can be input into the association algorithm, and the first feature information can be determined by the association algorithm.

[0085] For another example, in practical applications, suppose we detect 3 targets at t and 4 targets at t+1: the character feature information of t is:

[0086] Goal 1: [Feature 1.1, Feature 1.2]

[0087] Goal 2: [Feature 2.1, Feature 2.2]

[0088] Objective 3: [Feature 3.1, Feature 3.2]

[0089] The object feature information at t+1 is:

[0090] Target A: [Feature A.1, Feature A.2]

[0091] Target B: [Feature B.1, Feature B.2]

[0092] Objective C: [Feature C.1, Feature C.2]

[0093] Objective D: [Feature D.1, Feature D.2]

[0094] By calculating the similarities between these targets, we can obtain a 3×4 association matrix containing the similarity scores between each target, which is convenient for subsequent matching and tracking.

[0095] In this embodiment, by constructing a character time relationship matrix and calculating a similarity score, each panoramic image frame captured by the main panoramic camera can be accurately analyzed, thereby accurately identifying the first feature information and ensuring the accuracy of subsequent target tracking.

[0096] In one embodiment, Figure 5 As shown, the target person captured by the secondary panoramic camera is tracked according to the first cluster center and the second cluster center, including:

[0097] S502: Receive first time information corresponding to the first cluster center in each frame of a first panoramic image sent by the main panoramic camera, where the first panoramic image is an image captured by the main panoramic camera.

[0098] Specifically, since the main panoramic camera shoots a panoramic video during shooting, the panoramic video includes multiple frames of panoramic images. Therefore, each frame of the first panoramic image usually has its corresponding first cluster center. The first time information corresponding to the first cluster center in each frame of the first panoramic image can be determined.

[0099] S504: Determine second time information corresponding to the second cluster center in each frame of a second panoramic image, where the second panoramic image is an image captured by the secondary panoramic camera.

[0100] Specifically, the same secondary panoramic camera also captures a panoramic video when capturing a panoramic video. Therefore, each frame of the second panoramic image will also have a corresponding second cluster center. The second time information corresponding to each frame of the second panoramic image can be determined.

[0101] S506: construct a first association matrix according to the first cluster centers and the first time information, and construct a second association matrix according to the second cluster centers and the second information.

[0102] S508, calculating the similarity between the first incidence matrix and the second incidence matrix, screening the second incidence matrix according to the similarity, and determining a target element in the second incidence matrix.

[0103] Specifically, a first association matrix may be constructed according to the first cluster center and the first time information. A second association matrix may be constructed according to the second cluster center and the second time. The similarity of each element in the first association matrix and the second association matrix may be calculated, and then the elements in the second association matrix may be screened using the similarity to obtain the target element.

[0104] In some exemplary embodiments, for example, the information in the first incidence matrix may generally include:

[0105] S1 element: [A1 first cluster center, B1 first time information]

[0106] S2 element: [A2 first cluster center, B2 first time information]

[0107] For example, the information in the second incidence matrix may include:

[0108] W1 element: [C1 second cluster center, D1 second time information]

[0109] W2 element: [C2 second cluster center, D2 second time information]

[0110] W3 element: [C3 second cluster center, D3 second time information]

[0111] W4 element: [C4 second cluster center, D4 second time information]

[0112] The similarity between W1 element to W4 element and S1 and S2 can be calculated respectively. For example, the similarity between W1 element and S1 element is 0.8, the similarity between W2 element and S1 element is 0.45, the similarity between W3 element and S1 element is 0.2, the similarity between W4 element and S1 element is 0.3, the similarity between W2 element and S1 element is 0.75, the similarity between W2 element and S2 element is 0.8, the similarity between W3 element and S2 element is 0.95, and the similarity between W2 element and S2 element is 0.1. The first association matrix can be used as a standard. Since there are only two elements in the first association matrix, two elements also need to be screened out from the second association matrix. The two elements with the highest similarity to S1 and S2 can be selected as target elements according to the similarity. The element with the highest similarity to S1 is W1 element, and the element with the highest similarity to S2 is W3 element. Then W1 element and W3 element can be target elements.

[0113] S510: Determine a target person photographed by the secondary panoramic camera according to the target element, and perform target tracking on the target person.

[0114] Specifically, the second time information and the second cluster center may be determined according to the target element, and then the target person photographed by the secondary panoramic camera may be determined according to the determined second time information and the second cluster center, and the target person may be tracked.

[0115] In this embodiment, by using time information to generate an association matrix, the similarity between the first association matrix and the second association matrix is ​​calculated, which can ensure that time information is used as a judgment condition in the process of ultimately determining the target person, thereby ensuring the accuracy of identifying the target person.

[0116] In one embodiment, Figure 6 As shown, after the target person is tracked, the method further includes:

[0117] S602: Determine a plurality of panoramic images captured by the secondary panoramic camera and containing the target person.

[0118] S604, sending the multi-frame panoramic image containing the target person to the main panoramic camera to instruct the main panoramic camera to perform video editing based on the multi-frame panoramic image and the panoramic image containing the target person taken by the main panoramic camera to obtain a video containing the target person.

[0119] Specifically, after the secondary panoramic camera tracks the target person, the secondary panoramic camera can also send the captured multi-frame panoramic images (which can be panoramic videos) containing the target person to the primary panoramic camera. The user can perform video editing in the primary panoramic camera based on the multi-frame panoramic images containing the target person captured by the primary panoramic camera and the received panoramic images containing the target person, thereby producing a video containing the target person. The video usually records the wonderful moments of the person.

[0120] In this embodiment, the secondary panoramic camera can send the panoramic image containing the target person to the primary panoramic camera, so as to integrate the panoramic images of all the target persons and facilitate subsequent editing.

[0121] In one embodiment, before receiving the first feature information sent by the main panoramic camera, the method further includes:

[0122] A connection request is sent to the main panoramic camera, where the connection request is used to instruct the main panoramic camera to establish a connection with the secondary panoramic camera.

[0123] Specifically, the main panoramic camera and the secondary panoramic camera can detect within their signal range, and when other panoramic cameras (including the main panoramic camera and the secondary panoramic camera) are detected, a connection request can be sent to the other panoramic cameras. The connection can be a wireless connection, such as a Bluetooth connection, a WIFI connection, etc.

[0124] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0125] Based on the same inventive concept, the embodiment of the present disclosure also provides a multi-panoramic camera target tracking device for implementing the multi-panoramic camera target tracking method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more multi-panoramic camera target tracking device embodiments provided below can refer to the limitations of the multi-panoramic camera target tracking method above, and will not be repeated here.

[0126] In one embodiment, Figure 7 As shown, a multi-panoramic camera target tracking device 700 is provided, wherein the multi-panoramic camera includes: a main panoramic camera and a plurality of sub-panoramic cameras, wherein the main panoramic camera is installed on an amusement vehicle, and the device includes: an information receiving module 702, a matching person determination module 704, a center point determination module 706 and a target tracking module 708, wherein:

[0127] The information receiving module 702 is used to receive the first characteristic information sent by the main panoramic camera; the first characteristic information is the characteristic information of the target person riding the amusement vehicle photographed by the main panoramic camera;

[0128] A matching person determination module 704 is used to perform pedestrian re-identification on the photographed person according to the first feature information, and determine a matching person among the persons photographed by the secondary panoramic camera that matches the first feature information of the target person;

[0129] The center point determination module 706 is used to receive the first cluster center determined after clustering the first feature information sent by the main panoramic camera, and to determine the second feature information corresponding to the matching person and cluster the second feature information to determine the second cluster center;

[0130] The target tracking module 708 is used to track the target person captured by the secondary panoramic camera according to the first cluster center and the second cluster center.

[0131] In one embodiment of the device, the target tracking module 708 includes:

[0132] A first similarity calculation module, used to calculate the similarity between the first cluster center and the second cluster center;

[0133] A screening module, used for screening the second cluster centers according to the similarity and a preset similarity threshold, and determining a target cluster center among the second cluster centers;

[0134] The tracking submodule is used to determine target feature information matching the target cluster center, determine the target person photographed by the secondary panoramic camera according to the target feature information, and track the target person.

[0135] In one embodiment of the device, the device further comprises:

[0136] The first feature information determination module is used to determine the multiple frames of panoramic images taken by the main panoramic camera, extract the character feature information of the characters contained in each frame of the panoramic image; construct a character time relationship matrix based on the character feature information of the characters contained in each frame of the panoramic image and the time of each frame of the panoramic image; calculate the similarity score between the character feature information at different times in the character time relationship matrix; and determine the first feature information of the target person contained in the character time relationship matrix based on the similarity score and a predetermined association algorithm.

[0137] In one embodiment of the device, the target tracking module 708 includes:

[0138] a time information receiving module, configured to receive first time information corresponding to the first cluster center in each frame of a first panoramic image sent by the main panoramic camera, wherein the first panoramic image is an image captured by the main panoramic camera;

[0139] The time information determination module is used to determine the second time information corresponding to the second cluster center in each frame of the second panoramic image, where the second panoramic image is an image taken by the secondary panoramic camera.

[0140] The association matrix construction module is used to construct a first association matrix according to the first cluster center and the first time information, and to construct a second association matrix according to the second cluster center and the second information.

[0141] The second similarity calculation module is used to calculate the similarity between the first association matrix and the second association matrix, screen the second association matrix according to the similarity, and determine the target element in the second association matrix.

[0142] The person tracking module is used to determine the target person photographed by the secondary panoramic camera according to the target element, and to track the target person.

[0143] In one embodiment of the device, the device also includes: an image sending module, used to determine a multi-frame panoramic image containing the target person taken by the secondary panoramic camera; send the multi-frame panoramic image containing the target person to the main panoramic camera to instruct the main panoramic camera to perform video editing based on the multi-frame panoramic images and the panoramic image containing the target person taken by the main panoramic camera to obtain a video containing the target person.

[0144] In one embodiment of the device, the device further comprises:

[0145] The request sending module is used to send a connection request to the main panoramic camera, where the connection request is used to instruct the main panoramic camera to establish a connection with the auxiliary panoramic camera.

[0146] Each module in the above-mentioned multi-panoramic camera target tracking device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0147] In one embodiment, a computer device is provided. The computer device may be a terminal (eg, a panoramic camera). The internal structure diagram thereof may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a multi-panoramic camera target tracking method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0148] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present disclosure, and does not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0151] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0152] It should be noted that the panoramic images involved in the present disclosure (including panoramic images taken by the main panoramic camera and panoramic images taken by the secondary panoramic camera) are all information and data authorized by the user or fully authorized by all parties.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present disclosure may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided by the present disclosure may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited thereto.

[0154] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The above-described embodiments only express several implementation methods of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the attached claims.

Claims

1. A multi-panoramic camera target tracking method, characterized in that: The multi-panoramic camera includes: a main panoramic camera and a plurality of sub-panoramic cameras, the main panoramic camera is installed on the amusement vehicle, and the method includes: receiving first characteristic information sent by the main panoramic camera; the first characteristic information is characteristic information of a target person riding in the amusement vehicle photographed by the main panoramic camera; Re-identify the captured person according to the first feature information, and determine a matching person among the persons captured by the secondary panoramic camera that matches the first feature information of the target person; receiving a first cluster center determined after clustering the first feature information sent by the main panoramic camera, and determining second feature information corresponding to the matching person and clustering the second feature information to determine a second cluster center; According to the first cluster center and the second cluster center, target tracking is performed on the target person captured by the secondary panoramic camera.

2. The method according to claim 1, characterized in that The step of tracking the target person captured by the secondary panoramic camera according to the first cluster center and the second cluster center includes: Calculating the similarity between the first cluster center and the second cluster center; According to the similarity and a preset similarity threshold, the second cluster centers are screened to determine a target cluster center among the second cluster centers; Target feature information matching the target cluster center is determined, a target person photographed by the secondary panoramic camera is determined according to the target feature information, and target tracking is performed on the target person.

3. The method according to claim 1, characterized in that The first characteristic information is obtained by: Determine a plurality of panoramic images taken by the main panoramic camera, and extract character feature information of characters contained in each panoramic image; According to the character feature information of the characters contained in each frame of the panoramic image and the time of each frame of the panoramic image, a character time relationship matrix is ​​constructed; Calculating similarity scores between character feature information at different times in the character time relationship matrix; The first characteristic information of the target person contained in the person time relationship matrix is ​​determined according to the similarity score and a predetermined association algorithm.

4. The method according to claim 1, characterized in that The step of tracking the target person captured by the secondary panoramic camera according to the first cluster center and the second cluster center includes: receiving first time information corresponding to the first cluster center in each frame of a first panoramic image sent by the main panoramic camera, where the first panoramic image is an image captured by the main panoramic camera; Determining second time information corresponding to the second cluster center in each frame of a second panoramic image, where the second panoramic image is an image captured by the secondary panoramic camera; constructing a first association matrix according to the first cluster center and the first time information, and constructing a second association matrix according to the second cluster center and the second information; Calculating the similarity between the first incidence matrix and the second incidence matrix, screening the second incidence matrix according to the similarity, and determining a target element in the second incidence matrix; According to the target element, a target person photographed by the secondary panoramic camera is determined, and target tracking is performed on the target person.

5. The method according to claim 1, characterized in that After tracking the target person, the method further includes: Determine a plurality of panoramic images captured by the secondary panoramic camera and containing the target person; The multi-frame panoramic images containing the target person are sent to the main panoramic camera to instruct the main panoramic camera to perform video editing based on the multi-frame panoramic images and the panoramic images containing the target person taken by the main panoramic camera to obtain a video containing the target person.

6. The method according to claim 5, characterized in that Before receiving the first feature information sent by the main panoramic camera, the method further includes: A connection request is sent to the main panoramic camera, where the connection request is used to instruct the main panoramic camera to establish a connection with the secondary panoramic camera.

7. A multi-panoramic camera target tracking device, characterized in that: The multi-panoramic camera includes: a main panoramic camera and a plurality of sub-panoramic cameras, wherein the main panoramic camera is installed on the recreational vehicle, and the device includes: An information receiving module, used for receiving first characteristic information sent by the main panoramic camera; the first characteristic information is characteristic information of a target person riding in the amusement vehicle photographed by the main panoramic camera; a matching person determination module, configured to perform person re-identification on the photographed person according to the first feature information, and determine a matching person among the persons photographed by the secondary panoramic camera that matches the first feature information of the target person; a center point determination module, configured to receive a first cluster center determined after clustering the first feature information sent by the main panoramic camera, and to determine the second feature information corresponding to the matching person and cluster the second feature information to determine the second cluster center; A target tracking module is used to track the target person captured by the secondary panoramic camera according to the first cluster center and the second cluster center.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images