A method and system for space surveillance based on multi-view vision array

Abstract

The application discloses a kind of space monitoring method and system based on multi-view vision array, obtain the first arrangement position of multiple sets of multi-view vision array of the space to be monitored;Control multiple sets of multi-view vision array to collect image, according to the imaging overlap area of camera pair in each set of multi-view vision array, determine the baseline length of camera pair in each set of multi-view vision array based on first arrangement position, obtain the second arrangement position of multiple sets of multi-view vision array;With second arrangement position as space monitoring basis, select part or all single set of multi-view vision array in multi-view vision array as first shooting array strategy, determine the image acquisition depth information combination arrangement in each set of multi-view vision array in first shooting array strategy Generation first space monitoring strategy executes space monitoring.According to the method and system of the application, different depth of field of monitoring space has higher image acquisition accuracy, realizes the effective coverage and high-precision monitoring of the space to be monitored.

Description

Technical Field

[0001] This invention belongs to the field of space monitoring technology, specifically relating to a space monitoring method and system based on a multi-view visual array. Background Technology

[0002] To ensure safe navigation, modern ships deploy numerous cameras and intelligent image analysis systems inside their cabins to monitor critical compartments and equipment. This system detects anomalies and facilitates timely intervention; it is known as a ship cabin surveillance system. Current cabin surveillance systems mostly consist of single monocular cameras positioned at different locations within the cabin. Fixed-focus cameras are only suitable for specific depths of field, while zoom cameras can adjust with depth of field but have limited field of view. Furthermore, monocular cameras cannot obtain depth information, making it difficult to pinpoint the coordinates of cabin anomalies. Some of the latest ship cabin surveillance systems utilize binocular stereo vision; however, binocular vision has a fixed baseline length, which is relatively small, resulting in low depth resolution when the depth of field is large. Summary of the Invention

[0003] Therefore, the technical problem to be solved by this invention is that existing monocular or binocular vision systems are unable to effectively cover and obtain accurate depth information for different depths of field. This invention proposes a spatial monitoring method and system that adapts to changes in depth of field. For large depth-of-field ranges, a multi-view vision array is used, utilizing different baseline lengths between different camera pairs to achieve high image acquisition accuracy for different depths of field in the monitored space, thus achieving effective coverage and high-precision monitoring of the space to be monitored, especially the space of ship cabins. Addressing at least one of the above-mentioned deficiencies or improvement needs of the prior art, this invention provides a spatial monitoring method based on a multi-view vision array, characterized by the following steps:

[0004] Obtain the first arrangement position of multiple sets of multi-view visual arrays in the space to be monitored;

[0005] The system controls the acquisition of images by multiple sets of multi-view arrays, and determines the baseline length of the camera pairs in each set of multi-view arrays based on the first arrangement position according to the overlapping imaging area of ​​the camera pairs in each set of multi-view arrays to obtain the second arrangement position of the multiple sets of multi-view arrays.

[0006] Using the second arrangement position as the basis for spatial monitoring, select some or all of the single multi-view arrays in the multi-view array as the first shooting array strategy, and determine the combination and arrangement of the image acquisition depth information in each multi-view array in the first shooting array strategy to generate the first spatial monitoring strategy to perform spatial surveillance.

[0007] Furthermore, the method for obtaining the first arrangement position is as follows: obtaining the arrangement position of each single set of multi-view arrays corresponding to each monitoring subspace of the space to be monitored; obtaining the arrangement position between each single set of multi-view arrays.

[0008] Furthermore, the method also includes adjusting the first spatial monitoring strategy, adjusting the combination and arrangement of image acquisition depth information in each set of multi-view vision arrays, and then obtaining a second spatial monitoring strategy to perform spatial monitoring.

[0009] Furthermore, the adjustment of the first spatial monitoring strategy is based on the following information: selecting images from a portion of the cameras in each multi-view visual array of the first spatial monitoring strategy, determining whether there is any abnormal information in the space to be monitored, and if so, initiating the strategy adjustment.

[0010] Furthermore, the first shooting array strategy is as follows: based on one or more cameras in the single set of multi-view arrays and one or more cameras in other single sets of multi-view arrays, several groups are formed. The cameras in each group perform image acquisition at the same depth of field in the space to be monitored, and the cameras in different groups acquire images at different depths of field.

[0011] Furthermore, the second spatial monitoring strategy corresponds to the second imaging array strategy: select the acquisition camera with the largest abnormal area in the single set of multi-view arrays and form several groups with one or more cameras in other single sets of multi-view arrays. The cameras in each group perform image acquisition at the same depth of field in the space to be monitored, and the cameras in different groups acquire images at different depths of field to obtain information about the abnormal area.

[0012] Furthermore, the method for determining the baseline length of each camera pair within the multi-view vision array is as follows: based on its closest distance L to the monitored object. near And the farthest distance L far To determine, determine the shortest baseline length L min L min =k n L near / f,k n Determine the longest baseline length L for the camera's parallax of nearby pixels. max L max =k f L far / f, where f is the same focal length of the camera pair, k f This refers to the parallax of the camera at distant pixels.

[0013] This invention also discloses a space surveillance system based on a multi-view visual array, characterized in that the system comprises:

[0014] Central processing unit, distributed and connected to multiple sub-processing units;

[0015] Each sub-processing device controls a single set of multi-view vision arrays; the single set of multi-view vision arrays includes at least two cameras;

[0016] The sub-processing device acquires the first arrangement position of each single set of multi-view arrays and sends it to the central processing device; the central processing device determines the baseline length of the camera pair in each set of multi-view arrays based on the overlapping area of ​​the images acquired based on the first arrangement position, acquires the second arrangement position of the multi-view arrays, uses the second arrangement position as the basis for spatial monitoring, selects some or all of the single sets of multi-view arrays in the multi-view arrays as the first shooting array strategy, determines the combination and arrangement of the image acquisition depth information in each set of multi-view arrays in the first shooting array strategy to generate the first spatial monitoring strategy to perform spatial monitoring;

[0017] The central processing unit adjusts the first spatial monitoring strategy, and after adjusting the combination and arrangement of image acquisition depth information in each set of multi-view vision arrays, obtains the second spatial monitoring strategy and sends it to the corresponding sub-processing unit to perform spatial monitoring.

[0018] The present invention discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0019] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the above-described method.

[0020] In summary, compared with the prior art, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0021] Multiple sets of multi-view visual arrays are used, and camera pairs with different baseline lengths are selected according to different depths of field to ensure the accuracy of depth information in a large monitoring space. Different camera pair selection methods are used when the cabin is normal or abnormal to further enhance the accuracy of depth information in the region of interest. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the spatial surveillance method based on a multi-view visual array implemented according to the present invention.

[0023] Figure 2 This is a schematic diagram of the actual configuration of a space surveillance system based on a multi-view visual array implemented according to the present invention.

[0024] Figure 3This is a schematic diagram illustrating the determination of the camera baseline of a multi-view visual array in a spatial surveillance method based on a multi-view visual array implemented according to the present invention.

[0025] Figure 4 This is a schematic diagram illustrating the typical single-set multi-view array arrangement principle of the spatial surveillance method based on multi-view array implemented according to the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0027] The multi-view vision array involved in this invention is described below:

[0028] A single multi-view array refers to a system consisting of three or more cameras that can acquire image depth information and has a wider field of view than a single camera. Furthermore, the camera pairs composed of these three different cameras have baselines of different lengths, which can adapt to different depths of field and achieve optimal depth resolution at different depths of field.

[0029] A multi-view vision array is a space monitoring system composed of multiple sets of single multi-view vision arrays.

[0030] like Figure 1 As shown, the spatial surveillance method based on a multi-view visual array proposed in this invention includes the following steps:

[0031] Obtain the initial arrangement positions of multiple sets of multi-view visual arrays in the space to be monitored;

[0032] Control multiple sets of multi-view vision arrays to acquire images, and determine the baseline length of the camera pairs in each set of multi-view vision arrays based on the imaging overlap area of ​​the camera pairs in each set of multi-view vision arrays and the first arrangement position to obtain the second arrangement position of the multiple sets of multi-view vision arrays.

[0033] Using the second deployment position as the basis for spatial monitoring, select some or all of the single multi-view arrays in the multi-view array as the first shooting array strategy, and determine the combination and arrangement of the image acquisition depth information in each multi-view array in the first shooting array strategy to generate the first spatial monitoring strategy to perform spatial surveillance.

[0034] This invention provides a spatial surveillance method based on a multi-view visual array, specifically for a large-scale ship cabin space, comprising the following steps:

[0035] Step 1: For typical large-scale ship cabin spaces, multiple multi-view vision arrays can be deployed around the fore and aft positions on both sides of the cabin. Each multi-view vision array has an independent vision processing device, and the vision processing devices are interconnected via a high-speed network. A central processing unit is also provided to process the overall information. A typical system example is... Figure 2 As shown.

[0036] Step 2, design of a single multi-view vision array, with different baseline lengths for each camera pair, based on their closest distance L to the monitored object. near And the farthest distance L far To determine the distance, the imaging plane of the multi-view vision array camera is used as the reference plane. The closest distance refers to the closest perpendicular distance of the monitored object to the reference plane, and the farthest distance refers to the farthest perpendicular distance of the monitored object to the reference plane. These distances can be measured and determined in advance. The baseline length refers to the distance between cameras, and the shortest baseline length refers to the shortest distance between any two cameras in the multi-view vision array. For simplification, it is assumed that all cameras have the same focal length f, where the closest distance L... near Used to determine the shortest baseline length L between camera pairs min L min =k n L near / f, where f is the camera focal length, k n This represents the near-field pixel disparity for both eyes, with a possible value of 300 and a maximum distance L. far Used to determine the longest baseline length L between camera pairs in a multi-view vision array. max ,

[0037] L max =k f L far / f, where f is the focal length and k f This represents the parallax of distant pixels in both eyes, and can be set to a value of 10, such as... Figure 3 As shown.

[0038] Step 3: A typical multi-view vision array can be a row of cameras horizontally or arranged in a matrix pattern. The spacing between the nearest neighbor cameras can be unequal or equal. A typical matrix (m rows and n columns) with equal spacing L... min Arrangement of multi-view visual arrays such as Figure 4 As shown.

[0039] Step 4: When the multi-view vision array captures image pairs, in order to obtain a better image matching effect, there should be a suitable overlap area between the image pairs. The typical overlap range is about 1 / 3 to 1 / 2 of the image area. The baseline length between cameras is optimized and adjusted according to the imaging overlap area.

[0040] Step 5: To obtain higher depth information accuracy for different depths of field, the multi-view vision array uses camera pairs with different baseline lengths for image matching and depth information calculation. Once the shortest baseline length is determined and the multi-view vision array is formed, the camera arrangement is no longer optimized. However, the array contains multiple combinations of camera pairs, and different baseline length camera pairs are selected according to different depth information ranges.

[0041] Step 6: Using a single multi-view vision array, select 2-3 monocular cameras as surveillance cameras and use intelligent image monitoring algorithms to monitor the scene to determine if there are any anomalies in the ship's cabins. The monitoring system will subsequently adopt different operating modes depending on whether the cabin scene is normal or abnormal. Based on existing cameras, only software preprocessing functions for images are added; a software image preprocessing step is performed; normal and abnormal modes are distinguished; when moving objects or local deformations are observed, an anomaly is considered; if an anomaly occurs, the system enters an anomaly monitoring mode; to improve the monitoring accuracy of abnormal areas, under normal circumstances, the entire area is considered evenly, with depth accuracy evenly distributed. In abnormal situations, to improve the accuracy of the abnormal area, the optimal camera pair for observing the abnormal area is selected, while the requirements for non-abnormal areas can be reduced.

[0042] Step 7: When the cabin scene is normal, the monitoring system is mainly used to obtain high-precision depth information. A typical camera pair selection method is: select camera pair (C1, C2) for the closest scene, (C1, C3) for the next closest scene, and so on, up to the furthest scene (C1, Cn); another typical camera pair selection method is: select camera pair (C1, C2) for the closest scene, (C2, C4) for the next closest scene, and then (C3, C6) for the furthest scene, and so on.

[0043] Step 8: When an anomaly occurs in the cabin scene, the selected 2-3 monocular cameras are used as surveillance cameras. Using the camera Cx with the largest detected anomaly area as a benchmark, the camera Cx is selected to be paired with camera Cx and has the best overlap matching area (the image areas of the two cameras overlap by 1 / 3 to 1 / 2; if less than 1 / 3, the camera with the largest overlap area is selected). x+y To form the optimal camera pair (C x C x+y This allows for the acquisition of optimal depth information for the anomaly region. A method for optimizing camera C... x+y The search method can perform a binary search in both directions C1 and Cn to quickly obtain the best overlapping matching region camera C. x+y Find the best camera pair (C x C x+y To obtain scene range and depth information, including anomalous regions, a typical camera pair selection is divided into two directions, C1 and Cn, using Cx as the reference. The direction towards C1 is: (C x C x+y), (C x-(y-1) C x ), (C x-(y-1)-(y-2) C x-(y-1) ), and so on, in the direction of Cn: (C x C x+y ), (C x+y C x+y+(y+1) ), (C x+y+(y+1) C x+y+(y+1)+(y+2) ), and so on.

[0044] Step 9: In the case of multiple sets of multi-view arrays, since the middle part of the cabin of a large ship is far away from the multi-view arrays placed at both ends, the depth information of the middle 1 / 3 area of ​​the cabin is averaged by the depth information of the two sets of multi-view arrays placed at the front and rear. Specifically, each set of multi-view arrays calculates the result independently, and the average of all calculation results is taken to obtain a more stable and reliable depth value.

[0045] The present invention further proposes a space surveillance system based on a multi-view vision array, comprising:

[0046] Central processing unit, distributed and connected to multiple sub-processing units;

[0047] Each sub-processing device controls a single multi-view vision array; a single multi-view vision array includes at least two cameras;

[0048] The sub-processing device acquires the first arrangement position of each single set of multi-view vision array and sends it to the central processing device; the central processing device determines the baseline length of the camera pair in each set of multi-view vision array based on the overlapping area of ​​the images acquired based on the first arrangement position, acquires the second arrangement position of the multiple sets of multi-view vision arrays, uses the second arrangement position as the basis for spatial monitoring, selects some or all of the single sets of multi-view vision arrays in the multi-view vision array as the first shooting array strategy, determines the combination and arrangement of the image acquisition depth information in each set of multi-view vision arrays in the first shooting array strategy to generate the first spatial monitoring strategy and executes spatial monitoring;

[0049] The central processing unit adjusts the first spatial monitoring strategy, and after adjusting the combination and arrangement of image acquisition depth information in each set of multi-view vision arrays, obtains the second spatial monitoring strategy and sends it to the corresponding sub-processing unit to perform spatial monitoring.

[0050] The adjustment of the first spatial monitoring strategy is based on the following information: Select some cameras in each multi-view visual array of the first spatial monitoring strategy to collect images, determine whether there is abnormal information in the space to be monitored, and if so, initiate strategy adjustment.

[0051] The first imaging array strategy is as follows: based on one or more cameras in a single multi-view array and one or more cameras in other single multi-view arrays, several groups are formed. The cameras in each group perform image acquisition at the same depth of field in the space to be monitored, and the cameras in different groups acquire images at different depths of field.

[0052] The second spatial monitoring strategy corresponds to the second imaging array strategy: select the acquisition camera with the largest abnormal area in a single set of multi-view arrays and form several groups with one or more cameras in other single sets of multi-view arrays. The cameras in each group perform image acquisition at the same depth of field in the space to be monitored, and the cameras in different groups acquire images at different depths of field to obtain information about the abnormal area.

[0053] To address the challenge of effectively covering and obtaining accurate depth information for ship cabin monitoring at varying depths, existing monocular or binocular vision systems struggle. This paper proposes a ship cabin monitoring method and system that adapts to changes in depth of field. It employs multiple multi-view vision arrays and selects camera pairs with different baseline lengths based on varying depths of field to ensure accurate depth information for large-area cabin monitoring. Furthermore, different camera pair selection methods are used when the cabin is in normal or abnormal condition to further enhance the accuracy of depth information in the region of interest.

[0054] The description in this specification is merely illustrative of the invention. Those skilled in the art can make various modifications or additions to the specific embodiments described or use similar methods to replace them, as long as they do not deviate from the content of this specification or exceed the scope defined in the claims, they should all fall within the protection scope of this invention.

Claims

1. A method of space surveillance based on multi-view vision array, characterized in that, Includes the following steps: Obtain the placement of multiple multi-view vision arrays in the space to be monitored; The system controls the acquisition of images by the multiple sets of multi-view arrays. Based on the overlapping imaging area of ​​the camera pairs in each set of multi-view arrays and the arrangement position, the baseline length of the camera pairs in each set of multi-view arrays is adjusted to obtain the updated arrangement position of the multiple sets of multi-view arrays. Based on the updated layout, a spatial monitoring strategy is generated by combining and arranging the image acquisition depth information of each multi-view array in the first shooting array strategy to perform spatial monitoring. The first shooting array strategy is as follows: one or more cameras in the single multi-view array are combined with one or more cameras in other single multi-view arrays to form several groups. The cameras in each group perform image acquisition at the same depth of field in the space to be monitored, and the cameras in different groups acquire images at different depths of field. Select a portion of cameras in each set of multi-view arrays in the first spatial monitoring strategy as monitoring cameras to acquire images. Use an intelligent image monitoring algorithm to monitor the scene to determine whether there is abnormal information in the space to be monitored. If so, initiate strategy adjustment. Adjust the first spatial monitoring strategy by adjusting the combination and arrangement of image acquisition depth information in each set of multi-view arrays. Then, obtain the second spatial monitoring strategy to perform spatial monitoring. The second spatial monitoring strategy corresponds to the second shooting array strategy: based on the monitoring camera with the largest detected abnormal area, find the acquisition camera with the best overlap matching area with the monitoring camera to form the best camera pair to obtain the optimal abnormal area depth information. The acquisition camera with the largest abnormal area in the single set of multi-view arrays is combined with one or more cameras in other single sets of multi-view arrays to form several groups. The camera pairs in each group perform image acquisition at the same depth of field in the space to be monitored. Cameras in different groups acquire images at different depths of field to obtain information about the abnormal area. For the depth information of the middle third area of ​​the ship's cabin, the depth information of the two sets of multi-view arrays arranged at the front and rear is averaged. Specifically, after each set of multi-view arrays independently calculates the depth information, all calculation results are averaged to obtain a more stable and reliable depth value.

2. The spatial surveillance method based on a multi-view visual array according to claim 1, characterized in that, The method for obtaining the arrangement positions of multiple sets of multi-view vision arrays in the space to be monitored is as follows: obtain the arrangement position of each single set of multi-view vision array corresponding to each monitoring subspace of the space to be monitored; Obtain the arrangement positions between each set of multi-view vision arrays.

3. The spatial surveillance method based on a multi-view visual array as described in claim 1 or 2, characterized in that, The method for determining the baseline length of each camera pair within the multi-view vision array is as follows: based on its closest distance to the monitored object. and the farthest distance To determine, determine the shortest baseline length , , Determine the longest baseline length for the camera's parallax of nearby pixels. , Where f is the same focal length of the camera pair, This refers to the parallax of the camera at distant pixels.

4. A space surveillance system based on a multi-view visual array, characterized in that, The system includes: a central processing unit, which is distributed and connected to multiple sub-processing units; each sub-processing unit controls a single multi-view vision array; the single multi-view vision array includes at least two cameras; The sub-processing device acquires the arrangement position of each single set of multi-view arrays and sends it to the central processing device. The central processing device adjusts the baseline length of the camera pairs in each set of multi-view arrays based on the overlapping area of ​​the images acquired based on the arrangement position, to obtain the updated arrangement position of the multi-view arrays. Using the updated arrangement position as the basis for spatial monitoring, the central processing device selects some or all of the single sets of multi-view arrays as the first shooting array strategy. It determines the combination and arrangement of the image acquisition depth information in each set of multi-view arrays in the first shooting array strategy to generate a first spatial monitoring strategy for spatial surveillance. The first shooting array strategy is as follows: one or more cameras in the single set of multi-view arrays are combined with one or more cameras in other single sets of multi-view arrays to form several groups. The camera pairs in each group perform image acquisition at the same depth of field in the space to be monitored, and the cameras in different groups acquire images at different depths of field. The central processing unit selects a portion of the cameras in each multi-view array of the first spatial monitoring strategy as monitoring cameras to acquire images. It uses an intelligent image monitoring algorithm to monitor the scene to determine whether there is abnormal information in the space to be monitored. If so, it initiates strategy adjustment. The first spatial monitoring strategy is adjusted by combining and arranging the image acquisition depth information in each multi-view array. Then, a second spatial monitoring strategy is obtained to perform spatial monitoring. The second spatial monitoring strategy corresponds to the second shooting array strategy: taking the monitoring camera with the largest detected abnormal area as a benchmark, it finds and pairs acquisition cameras with the best overlap matching area with the monitoring camera to obtain the optimal abnormal area depth information. The acquisition camera with the largest abnormal area in the single multi-view array is combined with one or more cameras in other single multi-view arrays to form several groups. The cameras in each group perform image acquisition at the same depth of field in the space to be monitored. Cameras in different groups acquire images at different depths of field to obtain information about the abnormal area.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.

Images