A global image target tracking method and system based on fast scene stitching

Abstract

The present invention provides a global image target tracking method and system based on fast scene splicing, including: obtaining block images of several consecutive moments of each area; for the block images of all areas at each moment, calling the topological relationship table and the topological relationship dictionary of each area, based on the topological relationship table, counting the number of adjacent but unspliced ​​areas in each unspliced ​​area, selecting the unspliced ​​area corresponding to the maximum value as the spliced ​​area, splicing the block images of the spliced ​​area and its adjacent areas according to the topological relationship dictionary of the spliced ​​area, deleting the topological relationship table of the spliced ​​area, judging whether there are still unspliced ​​areas, if so, reselecting the spliced ​​area in the unspliced ​​area, splicing the block images of the spliced ​​area and its adjacent areas, and deleting the topological relationship table of the spliced ​​area, until there are no more unspliced ​​areas, and obtaining a global image; based on the global images at all moments, using a tracking algorithm to track the target. The time used for splicing is shortened and the efficiency of splicing is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a global image target tracking method and system based on fast scene splicing. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of deep learning, image stitching algorithms are becoming increasingly mature. Many computer vision tasks are no longer satisfied with single-lens images, and are beginning to develop towards more difficult multi-lens and cross-lens tasks. This puts forward requirements for the rapidity of cross-lens stitching and global image target tracking. Summary of the invention

[0004] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a global image target tracking method and system based on fast scene stitching, which does not need to traverse all topological relationship data when obtaining the global image, shortens the time used for stitching, and improves the efficiency of stitching.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] A first aspect of the present invention provides a global image target tracking method based on fast scene stitching, comprising:

[0007] Obtain block images of each region at several consecutive moments;

[0008] For the block images of all regions at each moment, the topological relationship table and the topological relationship dictionary of each region are called, and based on the topological relationship table, the number of adjacent but unspliced ​​regions in each unspliced ​​region is counted, and the unspliced ​​region corresponding to the maximum value is selected as the spliced ​​region, and the block images of the spliced ​​region and its adjacent regions are spliced ​​according to the topological relationship dictionary of the spliced ​​region, and the topological relationship table of the spliced ​​region is deleted, and it is determined whether there are still unspliced ​​regions. If so, the spliced ​​region is reselected in the unspliced ​​region, and the block images of the spliced ​​region and its adjacent regions are spliced, and the topological relationship table of the spliced ​​region is deleted, until there are no more unspliced ​​regions, and a global image is obtained;

[0009] Based on the global image at all times, a tracking algorithm is used to track the target.

[0010] Furthermore, when there are multiple unjoined regions corresponding to the maximum value, an unjoined region corresponding to the maximum value is randomly selected as the joined region.

[0011] Furthermore, the key in the topological relationship dictionary of a region represents the label of the adjacent region of the region.

[0012] Furthermore, the values ​​in the topological relationship dictionary of a region represent the nodes of the common edges between the region and the adjacent regions.

[0013] Furthermore, each region has at least four nodes, and there are at least two identical nodes on the common edge between a region and its adjacent region.

[0014] A second aspect of the present invention provides a global image target tracking system based on fast scene stitching, comprising:

[0015] The block image acquisition module is configured to: acquire block images of each region at a plurality of consecutive moments;

[0016] A splicing module is configured to: for the block images of all regions at each moment, call the topological relationship table and the topological relationship dictionary of each region, count the number of adjacent but unspliced ​​regions in each unspliced ​​region based on the topological relationship table, select the unspliced ​​region corresponding to the maximum value as the spliced ​​region, splice the block images of the spliced ​​region and its adjacent regions according to the topological relationship dictionary of the spliced ​​region, delete the topological relationship table of the spliced ​​region, determine whether there are still unspliced ​​regions, and if so, reselect the spliced ​​region in the unspliced ​​region and splice it with the block images of the adjacent regions until there are no more unspliced ​​regions, thereby obtaining a global image;

[0017] The target tracking module is configured to: track the target using a tracking algorithm based on the global image at all times.

[0018] Furthermore, the key in the topological relationship dictionary of a region represents the label of the adjacent region of the region.

[0019] Furthermore, the values ​​in the topological relationship dictionary of a region represent the nodes of the common edges between the region and the adjacent regions.

[0020] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the global image target tracking method based on rapid scene stitching as described above.

[0021] A fourth aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in a global image target tracking method based on rapid scene stitching as described above are implemented.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] The present invention provides a global image target tracking method based on fast scene stitching, which collects cross-shot area data, applies the cross-shot area topological relationship logic to the scene stitching and target tracking algorithm, and proposes a stitching optimization algorithm, so that there is no need to traverse all the topological relationship data when obtaining the global image, shortens the time used for stitching, and improves the efficiency of stitching. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0025] Figure 1 This is a flow chart of a global image target tracking method based on fast scene stitching according to the first embodiment of the present invention;

[0026] Figure 2 Schematic diagram of a global image target tracking method based on fast scene stitching according to the first embodiment of the present invention;

[0027] Figure 3 Schematic diagram of the divided areas and nodes of the first embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of four-region splicing in Embodiment 1 of the present invention;

[0029] Figure 5 It is a schematic diagram of nine-area splicing according to the first embodiment of the present invention. DETAILED DESCRIPTION

[0030] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0031] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0032] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0033] Embodiment 1

[0034] This embodiment provides a global image target tracking method based on fast scene stitching, such as Figure 1 and Figure 2 As shown, the block images of several consecutive moments collected by the image acquisition device set in each area are obtained; for the block images of all areas at each moment, the topological relationship table and the topological relationship dictionary of each area are called, and based on the topological relationship table, the number of adjacent but unjoined areas in each unjoined area is counted, and the unjoined area corresponding to the maximum value is selected as the joined area, and the block images of the joined area and its adjacent areas are joined according to the topological relationship dictionary of the joined area, and the topological relationship table of the joined area is deleted, and the joined areas in the topological relationship table are deleted, and it is determined whether there are still unjoined areas. If so, the joined area is reselected in the unjoined area, and the block images of the joined area and its adjacent areas are joined, and the topological relationship table of the joined area is deleted until there are no more unjoined areas, and a global image is obtained; based on the global image at all moments, a tracking algorithm is used to track the target. Specifically, the following steps are included:

[0035] Step 1: Get the label and node of each area.

[0036] Specifically, the user's node marking instruction is obtained, and the image acquisition device of each area is controlled to acquire an image, that is, the block images acquired by each image acquisition device, and transmitted to the display device for display through the display device. The user labels the block images of each area and marks the nodes; the numbers and nodes marked by the user for the block images of each area are obtained, that is, the numbers and nodes for each area.

[0037] Specifically, in a certain scene, the user can define and divide the scene into several areas, respectively set up an image acquisition device in each area, adjust the angles of the image acquisition devices to be the same, and synchronize the time of data acquisition.

[0038] For example, the scene is divided into four areas by user-defined, and the block images collected by the image acquisition device in each area are marked with numbers 0, 1, 2, and 3 respectively, so the corresponding areas are numbered 0, 1, 2, and 3.

[0039] Each block image is marked with at least four nodes (i.e., at least four pixel positions in each block image are marked as nodes, and the block images collected by two image acquisition devices in adjacent areas have at least two identical nodes). The nodes are the boundary points of the marked area division, such as Figure 3 As shown in , N1, N2, etc. are recorded as the nodes of the labels of each region. That is, each region has at least four nodes, and there are at least two identical nodes on the common edge between a region and its adjacent regions.

[0040] The nodes of the block images collected by the image collection device in a certain area marked by the user are the nodes of all the block images collected by the image collection device in the area in the subsequent steps.

[0041] Step 2: Based on the label and nodes of each area, construct a topological relationship table and topological relationship dictionary between all areas.

[0042] The topological relationship refers to the connection and adjacency between image elements in space, and the shots can only specify whether they are adjacent. The shot information cannot be modeled as to which edge the captured regional images are adjacent to.

[0043] The topological relationship dictionary refers to the dictionary data that stores the topological relationship of the block area. The dictionary data consists of key-value pairs, and uses list and dictionary data structures to represent the topological relationship between different areas. Among them, the list data represents the topological adjacency relationship between different areas, the key in the dictionary data represents the label of the adjacent area, and the value in the dictionary represents the node information of the common edge of the adjacent area.

[0044] Specifically, for a region's topological relationship table, the number of its elements is the number of regions, and the element value represents the adjacency relationship with the remaining regions. Specifically, if there are at least two identical nodes between two different regions, the element representing the adjacency relationship between the two regions in the topological relationship table is 1, otherwise, it is -1.

[0045] For example, the length of the definition list is N, where N=4, the index value of the topological relationship table element represents the label of the corresponding area, and the element in the topological relationship table represents the adjacency relationship between the index value area and the current area, where "1" represents that the area is adjacent to the current area, "0" represents that the area is the current area, and "-1" represents that the area is not adjacent to the current area, such as Figure 3 As shown, taking area 0 as an example, its topological relationship list is [0,1,1,-1].

[0046] Each dictionary data records the labels and node information of a region and its adjacent regions. For example, the topological relationship dictionary of the region labeled 0 is {1: "N3N4", 2: "N1N4"}.

[0047] Step 3: Obtain block images at several consecutive moments acquired by an image acquisition device arranged in each area.

[0048] For example, the scene is divided into four regions, each region is marked with 0, 1, 2, and 3 respectively, and the obtained block image is recorded as Among them, t represents the time step of data acquisition, which can be understood as the t-th frame, that is, the t-th moment in a number of consecutive moments, and 0, 1, 2, and 3 represent regions. For example, Indicates the second frame image collected from the second area.

[0049] Step 4: stitching the block images collected in all regions. According to the topological relationship table and topological relationship dictionary obtained in step 2, the SURF (Speeded Up Robust Features) stitching algorithm is used to stitch the block images collected at the same time in step 3. Specifically:

[0050] Based on the topological relationship table, the number of adjacent but unjoined areas in the unjoined area is counted (that is, the number of elements with element value 1 in the topological relationship table corresponding to each unjoined area), and the stitching optimization algorithm is used to select the stitched area as area 0 (or area 1 area 2 or area 3). According to the corresponding topological relationship dictionary data in step 2, the key is traversed, and the value is further indexed. Then, the SURF stitching algorithm is called to stitch the block images of the stitched area and the corresponding adjacent area, and the topological relationship table of the stitched area is deleted. At the same time, the elements between the unjoined area and the stitched area in the topological relationship table corresponding to the unjoined area are changed to -1. When the traversal of the topological relationship dictionary data of the stitched area is completed, the stitched image of the stitched area and all the adjacent areas can be obtained, such as Figure 4 as shown in .

[0051] The splicing optimization algorithm is specifically as follows: using the list or dictionary data in step 2, count the number of adjacent but unjoined areas in the unjoined area (for the list, first, record the number of elements in the topological relationship list of the area with a value of 1, and the number of areas adjacent to the area but not joined is obtained; for the dictionary data, first, calculate the number of key-value pairs in the dictionary, which represents the number of areas adjacent to the area, and then subtract the areas that have been spliced ​​in the previous splicing, and the number of areas adjacent to the area but not joined is obtained; the definition of the elements in the list can tell the algorithm the area currently being spliced, while the dictionary data only contains the adjacency information of the area), the area with the largest number is used as the area to be spliced, and the optimization algorithm is called before each splicing to select the area to be spliced. After the area to be spliced ​​is selected, the SURF splicing algorithm is called at the same time for splicing according to its topological information with the adjacent area, and so on, until the global scene is obtained. Regarding the selection of the splicing area, such as Figure 4 As shown in , when stitching an image divided into four regions, any region 0, 1, 2, or 3 can be selected for stitching for the first time. Because at the initial moment, the number of adjacent non-stitched regions of each region is 2. When stitching for the second time, only one non-stitched region is left. Therefore, only two steps are needed to complete the stitching, as shown in Figure 4 As shown in Figure 5As shown in the figure, when the image divided into nine regions is stitched, region 4 is selected for the first time, because its adjacent non-stitched area at the initial moment is the maximum value of 4. When stitching for the second time, the number of adjacent non-stitched areas of the remaining four unstitched regions is 0, and the remaining four regions can be stitched in any order. Through the above steps, the images of the four regions collected at time t (0≤t≤100) can be Stitched into a global image Img t .

[0052] Step 5: The global image Img obtained after stitching t , the DeepSort tracking algorithm is used to achieve target tracking in the global image.

[0053] The present invention collects cross-lens area data and applies the cross-lens area topological relationship logic to scene stitching and target tracking algorithms. First, the global scene is divided into several areas and each area is calibrated, then the lens is set up to collect image data of each area, the angle of the lens setting is required to be the same, and the time of cross-lens data collection is synchronized, next, the boundary points of the divided areas are marked and the topological relationship between the areas is constructed, the constructed topological relationship is represented by a list and a dictionary data structure, and the block images collected at the same time are spliced ​​using a general splicing algorithm and a topological relationship data representation. During the splicing process, the proposed splicing optimization algorithm can be used, so that all the topological relationship data does not need to be traversed during the splicing process, which shortens the time used for splicing, and finally, the tracking algorithm is used to achieve target tracking for the spliced ​​global scene image. We encapsulate the algorithm for actual testing. The above method has the characteristics of being general and highly practical. The algorithm combines the regional topological relationship across shots with the stitching and target tracking algorithms, and applies the topological logic to the fast stitching algorithm of the scene. Specifically, it uses list and dictionary data structures to represent and store topological relationships such as adjacency and nodes between regions, and on this basis proposes stitching optimization to achieve fast scene stitching and target tracking.

[0054] Embodiment 2

[0055] This embodiment provides a global image target tracking system based on fast scene stitching, which specifically includes the following modules:

[0056] The block image acquisition module is configured to: acquire block images of each region at a plurality of consecutive moments;

[0057] A splicing module is configured to: for the block images of all regions at each moment, call the topological relationship table and the topological relationship dictionary of each region, count the number of adjacent but unspliced ​​regions in each unspliced ​​region based on the topological relationship table, select the unspliced ​​region corresponding to the maximum value as the spliced ​​region, splice the block images of the spliced ​​region and its adjacent regions according to the topological relationship dictionary of the spliced ​​region, delete the topological relationship table of the spliced ​​region, determine whether there are still unspliced ​​regions, and if so, reselect the spliced ​​region in the unspliced ​​region and splice it with the block images of the adjacent regions until there are no more unspliced ​​regions, thereby obtaining a global image;

[0058] The target tracking module is configured to: track the target using a tracking algorithm based on the global image at all times.

[0059] The key in the topological relationship dictionary of a region represents the label of the adjacent region of the region.

[0060] The values ​​in the topological relationship dictionary of a region represent the nodes of the common edges between the region and the adjacent regions.

[0061] It should be noted here that each module in this embodiment corresponds to each step in Example 1 one by one, and the specific implementation process is the same, which will not be repeated here.

[0062] Embodiment 3

[0063] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the global image target tracking method based on rapid scene stitching as described in the first embodiment above are implemented.

[0064] Embodiment 4

[0065] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in a global image target tracking method based on rapid scene stitching as described in the first embodiment above are implemented.

[0066] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0067] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0068] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0069] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0070] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0071] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A global image target tracking method based on fast scene stitching, characterized in that: include: Obtain block images of each region at several consecutive moments; For the block images of all regions at each moment, the topological relationship table and the topological relationship dictionary of each region are called, and based on the topological relationship table, the number of adjacent but unjoined regions in each unjoined region is counted, and the unjoined region corresponding to the maximum value is selected as the joined region, wherein the number of elements in the topological relationship table of a region is the number of regions, and the element value represents the adjacency relationship with the remaining regions; According to the topological relationship dictionary of the spliced ​​area, the block images of the spliced ​​area and its adjacent areas are spliced, and the topological relationship table of the spliced ​​area is deleted, and it is determined whether there are still unspliced ​​areas. If so, the spliced ​​area is reselected from the unspliced ​​area, and the block images of the spliced ​​area and its adjacent areas are spliced, and the topological relationship table of the spliced ​​area is deleted until there are no more unspliced ​​areas, and a global image is obtained; wherein the key in the topological relationship dictionary of a region represents the label of the adjacent area of ​​the region; and the value in the topological relationship dictionary of a region represents the node of the common edge of the region and the adjacent area; Based on the global image at all times, a tracking algorithm is used to track the target.

2. A global image target tracking method based on fast scene stitching as claimed in claim 1, characterized in that: When there are multiple unjoined regions corresponding to the maximum value, an unjoined region corresponding to the maximum value is randomly selected as the joined region.

3. A global image target tracking method based on fast scene stitching as claimed in claim 1, characterized in that: Each region has at least four nodes, and there are at least two identical nodes on the common edges between a region and its adjacent regions.

4. A global image target tracking system based on fast scene stitching, characterized in that: include: The block image acquisition module is configured to: acquire block images of each region at a plurality of consecutive moments; The splicing module is configured to: for the block images of all regions at each moment, call the topological relationship table and the topological relationship dictionary of each region, count the number of adjacent but unspliced ​​regions in each unspliced ​​region based on the topological relationship table, and select the unspliced ​​region corresponding to the maximum value as the spliced ​​region, wherein the number of elements in the topological relationship table of a region is the number of regions, and the element value represents the adjacency relationship with the remaining regions; According to the topological relationship dictionary of the spliced ​​area, the block images of the spliced ​​area and its adjacent areas are spliced, and the topological relationship table of the spliced ​​area is deleted to determine whether there are still unspliced ​​areas. If so, the spliced ​​area is reselected from the unspliced ​​area and spliced ​​with the block images of the adjacent areas until there are no more unspliced ​​areas, thereby obtaining a global image; wherein the key in the topological relationship dictionary of a region represents the label of the adjacent area of ​​the region; and the value in the topological relationship dictionary of a region represents the node of the common edge between the region and the adjacent area; The target tracking module is configured to: track the target using a tracking algorithm based on the global image at all times.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in a global image target tracking method based on fast scene stitching as described in any one of claims 1 to 3 are implemented.

6. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the global image target tracking method based on fast scene stitching as described in any one of claims 1 to 3 are implemented.

Images