Camera laying method in semi-closed scene

Abstract

The invention discloses a camera laying method in a semi-closed scene, and the method comprises the following steps: constructing a three-dimensional model of the semi-closed scene, and obtaining an entrance and exit region and a shunting region; selecting an installable position of the entrance and exit area to perform simulation layout based on a camera coverage area to obtain a plurality of first candidate layout point positions; coordinate mapping analysis based on identification errors is carried out on the camera of each first candidate layout point location; according to the mapped position error, selecting an optimal layout point location covering the maximum vehicle bounding box; and if the sensing range of the optimal layout point location of the entrance and exit region cannot cover the whole semi-closed scene, carrying out view field overlapping point location layout from the edge to the interior from the entrance and exit region to the shunting region. The method has the beneficial effects that the matching between the camera resource distribution and the tracking demand is ensured through the priority coverage analysis of the key area, and the blind area and redundancy problems existing in the traditional empirical point distribution are effectively avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation, in particular to a camera layout method in a semi-closed scene. BACKGROUND

[0002] The semi-closed scene refers to a space scene in which part of the space is closed and part of the elements are allowed to communicate with the outside world. Common semi-closed scenes include service areas and parking lots, and other regional scenes including entrances and exits.

[0003] Taking a service area as an example, a highway service area is a key node in the transportation system, which undertakes multiple functions such as temporary parking of vehicles, energy supply, and emergency management. In order to realize the operation management and safety protection of the service area, a large number of video monitoring cameras are usually laid out during the design and construction stage. With the development of computer vision and intelligent video analysis technology, using cameras for global tracking of dynamic targets has gradually become a feasible solution.

[0004] However, the current camera layout strategy is mainly oriented towards "area monitoring", and lacks systematic support for vehicle "cross-camera, full-track" tracking requirements. Especially in semi-closed spaces such as service areas, traditional solutions fail to fully consider the spatial continuity of camera fields of view, the accuracy of frame mapping, and the integrity of cross-angle target matching, resulting in the following problems in tracking effect: Spatial coverage is not continuous: traditional layout is mainly based on monitoring blind area filling and local coverage requirements, lacking planning for the continuity of camera fields of view, which can easily cause monitoring blind areas or tracking link breaks.

[0005] Frame mapping is not accurate: fisheye, wide-angle, and other cameras are often used to expand the coverage range, but such lenses have severe distortion, making it difficult to establish an accurate mapping between the image and the actual coordinates, affecting three-dimensional reconstruction and multi-view matching.

[0006] Tracking ability is weakened: since target recognition is mainly based on the local coordinate system of the camera, cross-camera recognition relies on accurate splicing and time synchronization between frames. In the case of discontinuous recognition areas or angle occlusion, it is easy to cause target tracking interruption. SUMMARY

[0007] One of the purposes of the present application is to provide a camera layout method in a semi-closed scene that can solve at least one of the defects in the background art.

[0008] To achieve the above at least one purpose, the technical solution adopted by the present application is: a camera layout method in a semi-closed scene, comprising the following steps: S100: constructing a three-dimensional model of the semi-closed scene and performing regional identification and division to obtain an entrance and exit area and a shunt area; S200: Select the installable position of the entrance and exit area to carry out camera coverage area-based simulation layout, and obtain a plurality of first candidate layout points with the largest coverage range of the entrance and exit area; S300: For the camera of each first candidate layout point, a plurality of vehicle bounding boxes based on recognition error are constructed and mapped from the camera coordinate system to the three-dimensional model coordinate system; S400: According to the mapped position error, the first candidate layout point capable of recognizing the most vehicle bounding boxes in the coverage range is selected as the best layout point of the entrance and exit area; S500: If the perception range of the best layout point of the entrance and exit area cannot cover the entire semi-closed scene, the layout of the visual overlap point from the edge to the inside is carried out from the entrance and exit area to the distribution area.

[0009] Preferably, in step S100, the obtained three-dimensional model body is maintained as a three-dimensional basic model and a cubic grid model with a specified specification; then in step S200, the process of obtaining the first candidate layout point through simulation layout is as follows: according to the installation height, horizontal installation angle, pitch angle of the camera and the installation position of the camera on the ground, the extrinsic matrix of the camera is calculated; for each installable point of the entrance and exit area, a vector from the pixel coordinate in the camera coordinate system to the direction of the camera is constructed as a pixel ray; the pixel ray is mapped from the camera coordinate system to the world coordinate system and the mapping intersection with the three-dimensional basic model is solved, to obtain a mapping intersection set that can be captured by all pixel rays; the convex hull of the effective mapping intersection is calculated, and the convex hull region is mapped to the cubic grid model to obtain a voxel grid set covered by the camera; the installable points under a plurality of installation parameter combinations with the largest number of covered voxel grids are taken as the first candidate layout points.

[0010] Preferably, when selecting the installable points of the entrance and exit area, the following constraint conditions exist for the field of view of the camera: the field of view needs to completely cover the single-lane path of the entrance and exit area; the field of view needs to completely cover the distribution node at the junction of the entrance and exit area and the distribution area; the field of view needs to completely cover the initial part of the multi-lane area in the distribution area.

[0011] Preferably, step S300 includes the following process: based on the structural size of the vehicle, a bounding box of the vehicle is constructed; the position of the center point of the bounding box of the vehicle in the camera coordinate system is projected to the image plane coordinate system, and the predicted center point of the bounding box of the vehicle is obtained by introducing the recognition error; based on the intrinsic parameter and installation pose of the camera, the predicted center point of the bounding box of the vehicle is back-projected to the three-dimensional model coordinate system.

[0012] Preferably, in step S400, the calculation formula of the mapped position error ε is as follows: ; Wherein, P represents a coordinate transformation function, (u0, v0) represents a world coordinate point coordinate of a vehicle bounding box center point after being back projected from an image plane coordinate system through internal and external parameters to the ground, s and t respectively represent pixel offset amounts in the horizontal and vertical directions of the image due to recognition errors, Δx and Δy respectively represent theoretical pixel offset amounts in the horizontal and vertical directions of the image due to recognition errors.

[0013] Preferably, the selection of the optimal layout point in step S400 includes the following process: in the voxel grid range covered by each first candidate layout point, all possible vehicle position voxels are traversed, and the position error ε corresponding to all vehicle position voxels is calculated; an error threshold is set, and all vehicle position voxels with a position error ε less than the set error threshold are marked as effective coverage voxels; the effective coverage voxels of all first candidate layout points are counted, and the first candidate layout point with the most effective coverage voxels under the premise of full coverage of the key area is selected as the optimal layout point.

[0014] Preferably, after obtaining the optimal layout point of the entrance and exit area, the calculation of the perception range Ω A of the optimal layout point includes the following process: Based on the camera installation position coordinates [X A , Y A ] T of the optimal layout point, the polar angle θ i of the center point (x i , y i ) of each effective voxel grid covered by the optimal layout point is calculated, ; According to the obtained polar angle θ i , the central angle θ center of the entire effective voxel grid region covered by the optimal layout point is calculated; , ; According to the obtained central angle, the relative angle Δθ i of each effective voxel grid is calculated, and the maximum value Δθ max and the minimum value Δθ min of all relative angles Δθ i are taken to calculate the effective horizontal viewing angle α eff of the perception area; , α eff = Δθ max - Δθ min (> 0); According to the maximum distance R max and the minimum distance R min from the perception boundary to the optimal layout point, the effective perception radius Rcons = 0.5 (R min + R max ); The perception range of the optimal layout point is Ω A The expression is: ; Wherein, θ represents the polar angle corresponding to any coordinate position (x, y) in the perception range Ω A , m represents the number of effective voxel grids covered by the optimal layout point, and respectively represent the angle average value in the y direction and the x direction.

[0015] Preferably, the step S500 comprises the following process: selecting a plurality of second candidate layout points in the installable positions of the distribution area by the perception intersection and the maximum coverage principle; for the camera of each second candidate layout point, constructing a plurality of vehicle bounding boxes based on the recognition error and mapping from the camera coordinate system to the three-dimensional model coordinate system; selecting the second candidate layout point which can recognize the most vehicle bounding boxes in the coverage range and has an overlap with the coverage range of the previous layout point as the optimal layout point for layout.

[0016] Preferably, the second candidate layout point needs to meet: the perception range of the layout camera is directed to the centroid of the voxel set of the adjacent edge of the corresponding perception range of the previous optimal layout point; at the same time, the perception range of the layout camera has an overlap with the perception range of the previous optimal layout point, and the position error of the vehicle position voxel corresponding to the overlap area is less than the set error threshold.

[0017] Preferably, when setting the layout points of the distribution area, the visual penetration between each distribution sub-area in the distribution area is judged; for the adjacent distribution sub-areas which cannot be visually penetrated, the cameras are independently arranged for each distribution sub-area; for the adjacent distribution sub-areas which can be visually penetrated, it is judged whether the camera perception range of one of the distribution sub-areas can cover the other distribution sub-area; if it can cover, the other distribution sub-area does not need to add points, otherwise the other distribution sub-area needs to add points.

[0018] Compared with the prior art, the application has the following beneficial effects: (1) By analyzing the priority coverage of the key area, the camera resource distribution is matched with the tracking demand, and the "blind area" and "redundancy" problems existing in the traditional experience-based point layout are effectively avoided.

[0019] (2) Through the analysis of the visual field overlap between adjacent camera points, it is ensured that there is a reasonable visual field connection relationship between the cameras; it can be applied to complex scenes where the target motion path is not fixed, there are multiple turns and obstructions, effectively improving the cross-domain recognition and trajectory splicing accuracy of the target between multiple cameras. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The figure is a schematic diagram of the overall working steps of the present application. DETAILED DESCRIPTION

[0021] In the following, the present application will be further described in conjunction with specific embodiments. It should be noted that in the description of the present application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present description, the illustrative description of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in the present description.

[0022] In the description of the present application, it should be noted that for orientation words such as the terms "center", "transverse", "longitudinal", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc. The orientation and positional relationship shown in the drawing is based on the orientation or positional relationship shown in the drawing, and is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and cannot be understood as limiting the specific protection scope of the present application.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0024] In this application, unless specifically defined otherwise, the terms "mounting", "connected", "connection", "fixed", and the like are to be construed in a broad sense, for example, can be connected, can be detachable connection, or integrated; can be mechanical connection, can be electrical connection; can be directly connected, can be indirectly connected through an intermediate medium, can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0025] In this application, unless specifically defined otherwise, the first feature "on" or "under" the second feature can include that the first and second features are in direct contact, or that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, the first feature "on", "above" and "above" the second feature includes that the first feature is directly above and obliquely above the second feature, or only indicates that the horizontal height of the first feature is higher than that of the second feature. The first feature "under", "below" and "below" the second feature includes that the first feature is directly below and obliquely below the second feature, or only indicates that the horizontal height of the first feature is less than that of the second feature.

[0026] The terms "comprising" and "having" and any variations thereof in the specification and claims of this application are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0027] One of the preferred embodiments of the present application is shown as Figure 1 A camera layout method in a semi-closed scene, comprising the following steps: S100: Construct a three-dimensional model of the semi-closed scene and perform regional identification and division to obtain an entrance and exit area and a shunt area.

[0028] S200: Select an installable position of the entrance and exit area to perform simulation layout based on the camera coverable area to obtain a plurality of first candidate layout points with the largest coverage range for the entrance and exit area.

[0029] S300: For the camera at each first candidate layout point, construct a plurality of vehicle bounding boxes based on the identification error and map from the camera coordinate system to the three-dimensional model coordinate system.

[0030] S400: According to the mapped position error, select the first candidate layout point that can identify the most vehicle bounding boxes in the coverage range as the best layout point for the entrance and exit area.

[0031] S500: If the sensing range of the optimal layout point of the entrance and exit area cannot cover the entire semi-closed scene, then the visual overlap point layout from the entrance and exit area to the shunting area is performed from the edge to the interior.

[0032] It can be understood that the technical solution of the present application is mainly to divide the semi-closed scene into a three-dimensional space by grid, thereby clearly defining the target area range that needs to be tracked globally, and identifying the candidate layout area that can be used for camera installation. Considering that the entrance and exit of the semi-closed scene is generally a clear single-lane path, and the interior is generally a multi-lane shunting area, therefore, for the entire semi-closed scene, the path selection of the vehicle from the single-lane to the multi-lane is the focus of global tracking, so when setting the candidate layout area, the camera layout of the entrance and exit area is considered, and then the camera layout is started from the entrance and exit, and then pushed to the interior key path in sequence, to ensure complete trajectory acquisition in the early entry area.

[0033] Compared with the traditional method, the present application uses the three-dimensional space information of the semi-closed scene, combines the geometric characteristics of each area, establishes a spatial geometric relationship model, and scientifically evaluates the installation position, posture and visible area of each camera. Through the priority coverage analysis of the driving area, parking area and other key areas, it is ensured that the camera resource distribution matches the tracking demand, and the "blind area" and "redundancy" problems existing in the traditional experience type point layout are effectively avoided.

[0034] At the same time, the technical solution of the present application can be adapted to various semi-closed scenes, such as high-speed service areas, refueling areas, urban public parking lots, etc., and can realize the point layout of different types of cameras, including fisheye cameras, omnidirectional cameras and high-altitude gimbals. Through unified geometric modeling and parameter adaptation mechanism, it has good generalization ability and engineering reuse value. Even in scenes with significant differences in sensor types and installation conditions, the layout scheme that meets the sensing target can be quickly generated.

[0035] In the embodiment, when performing the above step S100, the three-dimensional model of the semi-closed scene can be constructed by the two-dimensional design drawing of the semi-closed scene. The three-dimensional model does not need to be modeled with high precision, and only needs to restore the spatial position and size information of each functional area in the scene according to the 1:1 scale; after the construction of the three-dimensional model is completed, the coordinate system of the model needs to be reconstructed to facilitate the subsequent unified camera layout point.

[0036] Specifically, taking the service area as an example in the semi-closed scene, the design drawing thereof should include the following key areas: a building area, a parking area, a vehicle lane area, a site boundary area, and a camera mountable area. The building area, the parking area, and the vehicle lane area can be used to determine the key monitoring and tracking areas. For example, the building area is usually not used as the main vehicle passing space, and thus can be excluded from the key tracking area. The identification of the camera mountable area is used to impose a layout constraint to ensure that the camera layout is engineering implementable, such as avoiding installation in the center of the motor vehicle lane, and preferentially selecting the grass area, the non-motor vehicle lane edge, or the building facade, and the like, which have a mounting basis, so as to reduce the electrical wiring complexity and improve the system deployment rationality.

[0037] It should be understood that after the three-dimensional model is constructed, the three-dimensional model can be maintained in two types; the first type is a three-dimensional base model M, and the second type is a voxelization method for spatial division of the three-dimensional model, which can form a cubic grid model M of a set specification. BEV When voxelizing the three-dimensional model, the specification of the cubic grid model can be selected by the person skilled in the art according to actual needs. For example, the grid specification of the unit cubic is 2 m x 2 m x h; wherein the horizontal and vertical resolutions are both 2 m, and the height h can be set as the full height of the semi-closed scene, that is, the cubic grid model is not set with upper and lower constraints, and is only used to establish a space envelope. After voxelizing the three-dimensional model, each cubic grid can be numbered.

[0038] Based on the above division of the cubic grid model, combined with the area attribute information of the original design drawing, the spatial voxel can be semantically classified, which can be mainly divided into the following three categories: (1) Key detection area; including the vehicle lane and the parking area, etc.

[0039] (2) Camera mountable area; including the green belt, the road edge, and the inside of the building and the corner, etc.

[0040] (3) Non-monitoring area; including the interior space of the building, the closed space, and the green belt, etc.

[0041] After the semantic classification is completed, all area label information can be projected and mapped into the three-dimensional space model, to provide a structural constraint condition for subsequent layout strategies.

[0042] In this embodiment, based on the three-dimensional base model M and the cubic grid model M obtained in step S100 BEV The process of obtaining the first candidate layout point by simulation layout in step S200 will be described in detail.

[0043] First, according to the installation height H of the camera, the horizontal installation angle , pitch angle and the installation position (x c , y c ) of the camera on the ground, the coordinates C of the optical center of the camera in the world coordinate system can be calculated: .

[0044] Then, the rotation matrix of the camera is constructed in the order of Euler angles; here, it can be specified that the rotation around the z-axis is the yaw angle , i.e., the horizontal installation angle, the rotation around the x-axis is the pitch angle , and the rotation around the y-axis is the roll angle ; therefore, the expressions of the rotation matrices R x , R y and R z in the z, y, z-axis directions are: .

[0045] .

[0046] .

[0047] The overall rotation matrix R of the camera can be obtained as: .

[0048] The translation vector t0 is obtained through the camera optical center position and the rotation relationship: t0=-RC; therefore, the extrinsic matrix of the camera can be obtained as: .

[0049] Then, because the pixel and frame coordinates of the camera are known, all pixel coordinates in the image of the camera can be obtained (without considering fisheye cameras or ultra-wide angles); here, it can be assumed that the ground scene covered by the camera is approximately a plane. It should be noted that in the process of obtaining the pixel coordinates, if the camera has distortion, the original pixels can be de-distorted to obtain using the distortion parameters.

[0050] Then, for each mountable point in the entrance and exit area, the vector from the pixel coordinates in the camera coordinate system to the direction of the camera is solved as the pixel light ray. The expression of the pixel light ray is: ; wherein K represents the intrinsic matrix.

[0051] Then, the pixel light ray is mapped from the camera coordinate system to the world coordinate system to obtain the world system ray in the world coordinate system The specific expression is as follows: ; wherein, is a light elongation parameter.

[0052] Then, the world system ray is solved, and the mapping intersection with the three-dimensional basic model M is obtained, to obtain all pixel rays The set of mapping intersections P0 that can be captured in the world coordinates is as follows: ; wherein, P i represents any mapping intersection in the set of mapping intersections P0.

[0053] It should be noted that if some world system rays do not generate mapping intersections with the three-dimensional basic model M, it indicates that the pitch angle of the camera corresponding to these world system rays is too small, causing the world system rays to fall outside the ground, and thus these points need to be deleted. Then, the remaining mapping intersections in the set of mapping intersections P0 are effective mapping intersections The specific expression is as follows: .

[0054] Then, the convex hull Conv( is calculated for the effective mapping intersections , and the convex hull region is mapped to the cubic grid model M BEV to obtain the set of voxel grids covered by the camera ; the expression of the set of voxel grids is as follows: ; wherein, represents any voxel grid covered by the camera, and G represents all voxel grids corresponding to the cubic grid model M BEV .

[0055] Finally, the installable points under multiple installation parameter combinations with the largest number of covered voxel grids are taken as the first candidate layout points.

[0056] It can be understood that the purpose of setting the first candidate layout points for the entrance and exit area is to identify the first split point of the vehicle on the entrance and exit path, that is, the node where the driving path of the vehicle changes from a single lane to multiple lanes. Therefore, when selecting the installable points for the entrance and exit area, the following constraints exist for the field of view of the camera: the field of view needs to completely cover the single-lane path of the entrance and exit area; the field of view needs to completely cover the split node at the junction of the entrance and exit area and the split area; and the field of view needs to completely cover the initial part of the multiple-lane area in the split area.

[0057] It should be known that the area A of a single voxel mesh v The area can be determined based on the specific specifications of the cube mesh model. Taking the aforementioned 2 m × 2 m × h specification as an example, the area of ​​a single voxel mesh is 4 m². 2 The camera's tilt angle The value is generally between 10° and 40°; the horizontal installation angle of the field of view parameter The value of is generally between 60° and 110°; in practical applications, cameras rarely rotate in the roll direction, so it can be assumed that the installation does not generate a roll angle for the camera. The value is 0. The installation height H of the camera can be determined according to the specific recognition accuracy of the camera. For example, a 2-megapixel camera is generally installed at a height of 4~6m, which can provide a large coverage area and high image quality.

[0058] After clarifying the relevant parameters, by enumerating the combinations of cameras at different pitch and yaw angles at each installable location in the entrance / exit area, the coverable area N under each angle combination can be calculated. c The value is selected; the installation points corresponding to the multiple angle combinations that cover the most target voxels are selected as the first candidate deployment points. The specific number of the first candidate deployment points, that is, the specific number of angle combinations that cover the most target voxels, can be selected according to the actual needs of those skilled in the art. For example, five, ten or fifteen first candidate deployment points can be selected; in this embodiment, ten are preferred.

[0059] It's important to note that cameras exhibit a perspective imaging characteristic in 3D scenes: pixels appear larger when closer to the camera and smaller when farther away. This means that in areas closer to the camera, the actual physical distance between adjacent pixels in the image is shorter, while in areas farther from the camera, the actual physical distance between pixels increases significantly. This phenomenon directly leads to a decrease in the positioning accuracy of objects in the image as distance increases. Therefore, after obtaining the first candidate deployment points using the camera's own parameters, vehicle tracking simulations can be performed on the coverage area corresponding to each first candidate deployment point. This involves simulating vehicle traffic at all locations within the coverage area where vehicles can pass. Then, based on the camera's recognition results under conditions of recognition error, the first candidate deployment point that can effectively identify the most vehicles is selected as the optimal deployment point.

[0060] Specifically, the vehicle recognition algorithm generates a bounding box (abbreviated as bbox) with reduced accuracy in areas far from the camera, which in turn causes the spatial positioning error to increase when the bbox is back-projected to the world coordinate system. This error will significantly interfere with cross-camera vehicle tracking, especially when multiple vehicles pass through the intersection area of two cameras at the same time. Insufficient accuracy in position matching can cause vehicle identification to be incorrectly matched, thereby disrupting the continuity of the tracking chain. Therefore, when performing vehicle tracking simulation for the first candidate layout point, the bounding box of the vehicle can be constructed based on the structural dimensions of the vehicle. The position of the center point of the vehicle bounding box in the camera coordinate system is projected onto the image plane coordinate system, and the predicted bounding box center point of the vehicle is obtained by introducing the recognition error. The predicted bounding box center point of the vehicle is back-projected to the three-dimensional model coordinate system based on the intrinsic parameters and installation pose of the camera.

[0061] Specifically, taking a small vehicle as a reference model, the dimensions of the small vehicle can be set as length = 3m, width = 1.5m, and height = 1.5m. In mainstream vehicle monitoring algorithms, the commonly used positive and negative sample division standard is that the IoU (Intersection over Union) of the pre-boundary box and the true boundary box should not be less than 0.8, i.e., the maximum recognition error of the vehicle boundary box is 20%. This error can be uniformly distributed in the horizontal and vertical directions of the image, and the expressions for the theoretical pixel offset Δx and Δy of the vehicle in the horizontal and vertical directions of the camera's perception image due to the recognition error are as follows: Δx = 0.1 × W img , Δy = 0.1 × H img . Where W img and H img are the width and height of the vehicle in the horizontal and vertical directions of the camera's perception image.

[0062] After obtaining the theoretical pixel offset in the horizontal and vertical directions of the vehicle camera's perception image, in order to facilitate the identification of the vehicle boundary box and subsequent statistics, the center point of the boundary box can be used instead of the vehicle position. The center point coordinates of the vehicle boundary box can be set as (x, y, z), and the coordinates after projection onto the image plane coordinate system are (u0, v0). Then, by introducing the recognition error, the coordinates of the predicted boundary box center point of the vehicle based on the recognition error can be obtained (u0±Δx, v0±Δy). Then, based on the intrinsic parameters and installation pose of the camera, the predicted boundary box center point of the vehicle can be back-projected to the three-dimensional space coordinate system, and the position error ε after error projection can be obtained. The calculation formula of the position error ε is as follows: .

[0063] wherein P represents a coordinate transformation function, s and t represent pixel offset amounts in horizontal and vertical directions of the image due to the recognition error, and Δx and Δy represent theoretical pixel offset amounts in horizontal and vertical directions of the image due to the recognition error.

[0064] It should be known that after obtaining the position error ε of the vehicle based on the recognition error, the obtained position error ε can be compared with a set error threshold, and if the calculated position error ε is less than the set error threshold, it indicates that the vehicle can be effectively captured by the camera, otherwise the vehicle can affect the subsequent global tracking due to the large recognition error. The specific value of the error threshold can be determined by the actual needs of those skilled in the art; for example, the error threshold can be selected as 1 m, that is, the position error ε of the vehicle collected by the camera data is less than 1 m, and it can be determined that the vehicle is effectively captured by the camera, otherwise the voxel corresponding to the position of the vehicle is removed from the coverable area N c

[0065] In this embodiment, the selection of the optimal layout point in step S400 includes the following process: in the voxel grid range covered by each first candidate layout point, all possible vehicle position voxels are traversed, and the position error ε corresponding to all vehicle position voxels is calculated; an error threshold is set, and all vehicle position voxels with a position error ε less than the set error threshold are marked as effective coverage voxels; the effective coverage voxels of all first candidate layout points are counted, and the first candidate layout point with the most effective coverage voxels under the premise of full coverage of the key area is selected as the optimal layout point.

[0066] For the convenience of understanding, the following can be described in detail through specific parameters.

[0067] Suppose the number of first candidate layout points is five, marked as #1 to #5; wherein the number of voxel grids corresponding to the coverable area of the first candidate layout point #1 is 30, the number of voxel grids corresponding to the coverable area of the first candidate layout point #2 is 32, the number of voxel grids corresponding to the coverable area of the first candidate layout point #3 is 34, the number of voxel grids corresponding to the coverable area of the first candidate layout point #4 is 34, and the number of voxel grids corresponding to the coverable area of the first candidate layout point #5 is 36. Assuming that one vehicle can be placed in every 2 voxel grids, the maximum number of vehicles that can pass through the coverable area of the first candidate layout point #1 at the same time is 15, the maximum number of vehicles that can pass through the coverable area of the first candidate layout point #2 at the same time is 16, the maximum number of vehicles that can pass through the coverable area of the first candidate layout point #3 at the same time is 17, the maximum number of vehicles that can pass through the coverable area of the first candidate layout point #4 at the same time is 17, and the maximum number of vehicles that can pass through the coverable area of the first candidate layout point #5 at the same time is 18.​

[0068] The position error calculation based on error identification is performed on the passable vehicles corresponding to the first candidate layout points #1 to #5 described above; if the number of vehicles meeting the error threshold requirement of the first candidate layout point #1 is 12, that is, the number of effective coverage voxels is 24; the number of vehicles meeting the error threshold requirement of the first candidate layout point #2 is 14, that is, the number of effective coverage voxels is 28; the number of vehicles meeting the error threshold requirement of the first candidate layout point #3 is 13, that is, the number of effective coverage voxels is 26; the number of vehicles meeting the error threshold requirement of the first candidate layout point #4 is 12, that is, the number of effective coverage voxels is 24; the number of vehicles meeting the error threshold requirement of the first candidate layout point #5 is 13, that is, the number of effective coverage voxels is 26. Since the number of effective coverage voxels corresponding to the first candidate layout point #2 is the largest, the first candidate layout point #2 can be taken as the best layout point.

[0069] In this embodiment, after the layout of the first camera layout point of the entrance and exit area of the semi-closed scene is completed, subsequent point layout can be performed by the layout mode of iterative deduction from the edge to the inside, so as to ensure the stable and continuous three-dimensional perception and cross-camera tracking capability of the whole space.

[0070] Specifically, the first camera layout point can be set as A, the position of which is P A , the orientation is θ A , and the effective sensing radius is R A . The sensing range Ω A of the point A can be regarded as a sector area with P A as the center and θ A as the orientation. The specific acquisition process of the sensing range Ω A of the point A is as follows: First, the coordinates of the position P A of the camera only take the plane components, that is, P A = .

[0071] Then, based on the calculation of the effective mapping intersection points, the center point set of the effective voxel grid covered by the camera is obtained as follows: ; wherein, represents the center point set, P i0 represents the center point of any effective voxel grid, m represents the total number of effective voxel grids, and (x i , y i ) represents the center point coordinates of any effective voxel grid.

[0072] Then, the polar angle θ i corresponding to each effective voxel grid center point is calculated, and the polar angle θ iThe calculation formula is as follows: .

[0073] Then, the central angle of all effective voxel mesh regions is calculated using the vector averaging method. : ; ;in, and These represent the average angles in the y and x directions, respectively.

[0074] Then, calculate the relative angle Δθ. i And based on the obtained relative angle Δθ i Calculate the maximum and minimum relative angles Δθ max and Δθ min .

[0075] .

[0076] Then, based on the obtained maximum and minimum relative angles Δθ max and Δθ min Conduct effective horizontal perspective The calculation of the width of the sector is expressed as follows: .

[0077] Among them, the effective horizontal perspective The effective interval is: .

[0078] After obtaining the angle information through the above process, it is also necessary to determine the effective sensing radius R. A Perform the calculation.

[0079] First, calculate the maximum and minimum distances from the camera's sensing boundary to the installation point. The specific expressions are as follows: .

[0080] Where, r i This represents the distance from the sensing boundary to the deployment point. R represents the coordinates of the boundary point of the perceived boundary. min and R max These represent the minimum distance and the maximum distance, respectively.

[0081] It's important to know that, theoretically, the error in vehicle recognition caused by a camera's field of view increases with distance; therefore, the camera's sensing radius R... A Theoretically, no sudden change will occur; therefore, for the sensing radius R A The effective value R can be taken. cons Effective radius Rcons = 0.5(R min + R max ).

[0082] The expression of the fan-shaped sensing range Ω A of the first camera is: .

[0083] Wherein, θ represents the polar angle corresponding to any coordinate position (x, y) in the sensing range Ω A .

[0084] It should be known that after the arrangement of the point A, there are two cases, one is that the sensing range of the point A has covered all the areas of the semi-closed scene; the other is that the sensing range of the point A cannot cover the shunt area or the subsequent road. Generally speaking, the range of the semi-closed scene is far beyond the sensing range of a single camera, so the first scenario basically does not appear, and the following will describe the specific process of adding camera point positions for the latter scenario.

[0085] In this embodiment, step S500 includes the following process: selecting a plurality of second candidate layout points in the installable positions of the shunt area through the sensing intersection and the maximum coverage principle. For the camera of each second candidate layout point, a plurality of vehicle bounding boxes based on the recognition error are constructed and mapped from the camera coordinate system to the three-dimensional model coordinate system. The second candidate layout point which can recognize the most vehicle bounding boxes in the coverage range and has an overlapping range with the coverage range of the previously laid point is selected as the best layout point for layout.

[0086] It can be understood that the specific selection process of the second candidate layout point is basically the same as that of the first candidate layout point, that is, the coverable area of the second candidate layout point can cover the most voxel grids, which meets the maximum coverage principle. At the same time, the coverable area of the second candidate layout point needs to have an overlapping area with the coverable area of the first candidate layout point, that is, both of them need to meet the sensing intersection principle. Specifically, for the second candidate layout point, the following needs to be met: the sensing range of the laid camera is directed to the centroid of the voxel set of the adjacent edge of the sensing range corresponding to the previous best layout point; assuming that the point of the first camera is A, the second candidate layout point is B (X B , Y B ), the centroid of the edge voxel set of the covered area of the first camera is a (X a , Y a ); then the expression of the orientation angle θ B of the second candidate layout point B is: Meanwhile, the sensing range of the camera arranged at the same time overlaps the sensing range of the previous best arrangement point, and the position error of the vehicle position voxel corresponding to the overlapping area is less than the set error threshold, that is, the minimum error principle.

[0087] It should be noted that when arranging the cameras in the diverging area, since there are multiple driving lanes in the diverging area, it is necessary to ensure that each driving lane can be effectively covered. However, based on the building scene in the diverging area, there are two cases. One is that the two adjacent driving lanes are separated by a higher object, such as a building, so that the field of view of the camera in one driving lane area cannot penetrate to the other driving lane. The other is that the two adjacent driving lanes are separated by a lower object, such as a green belt, so that the field of view of the camera in one driving lane area can penetrate to the other driving lane. For the convenience of understanding, the specific camera point arrangement process for the above two cases will be described in detail below.

[0088] In this embodiment, when arranging the point positions in the diverging area, the visual penetration between each sub-diverging area in the diverging area is judged. For adjacent sub-diverging areas that cannot be visually penetrated, cameras are independently arranged for each sub-diverging area. For adjacent sub-diverging areas that can be visually penetrated, it is judged whether the camera sensing range of one of the sub-diverging areas can cover the other sub-diverging area. If it can be covered, the other sub-diverging area does not need to add a point position, otherwise a point position needs to be added to the other sub-diverging area.

[0089] It can be understood that the visual penetration between two adjacent sub-diverging areas can be judged by the number of continuous non-passable voxels between them. If there are k continuous non-passable voxels between two adjacent sub-diverging areas, it can be determined that the two sub-diverging areas cannot be visually penetrated. The specific value of k can be set by the person skilled in the art according to actual needs.

[0090] If there are houses or high-density vegetation barriers between two adjacent sub-diverging areas, cameras can be independently arranged for each two adjacent sub-diverging areas. The specific arrangement logic is: taking the point position arranged at the entrance as the starting point, deducing along the driveway, and each point position satisfies the maximum coverage principle and the minimum error principle of continuous area. If two adjacent sub-diverging areas are separated by low green belts and other visually penetrable voxels, a joint arrangement strategy is adopted. The specific arrangement logic is: two adjacent sub-diverging areas can be set as D and S. If the camera is arranged in D, the camera needs to satisfy the maximum coverage principle and the minimum error principle of continuous area for area S.

[0091] It should be noted that when the camera is laid out, a long and narrow area may also be encountered. For the camera layout of the long and narrow area, it is necessary to meet: while ensuring that the perception range covers the vehicle entrance and exit communication area, the point layout is carried out in the left-right alternating manner along the long and narrow area. Through this layout method, the purpose of using the camera to track the whole vehicle can be met under the condition of global coverage of the camera.

[0092] For the convenience of understanding, the specific working process of the technical solutions of the present application will be described in detail below.

[0093] Firstly, the design drawings of the semi-closed scene are acquired to construct a three-dimensional model and perform meshing. According to the meshing result, the entrance and exit area and the shunting area are obtained through partition identification. Meanwhile, each area can be further subdivided into a key monitoring area, an installable camera area and a non-key monitoring area.

[0094] Then, through the calculation of the coverable area in the entrance and exit area, the best layout point of the camera A at the entrance and exit position can be determined. After the layout of the camera A is completed, whether the perception range of the camera A can cover the shunting area and the perception blind area of the shunting area by the camera A are identified. If the camera A cannot cover the shunting area, then the deduction of the cameras B, C, D, … in the shunting area from the entrance and exit area to the center of the semi-closed scene is carried out to obtain the layout main path of the cameras B, C, D, ….

[0095] When the cameras B, C, D, … are laid out, taking the camera B as an example, the overlapping verification of the coverable area of the cameras B and A is needed, and the error modeling of the overlapping area and the position error verification are needed. Meanwhile, it is also necessary to ensure the maximum area coverage of the camera B.

[0096] Meanwhile, when the cameras B, C, D, … are laid out, the shielding type of the adjacent shunting sub-area in the shunting area needs to be judged. If there is a shielding that cannot be visually penetrated, the independent layout of the camera for each shunting sub-area is carried out, otherwise the joint layout of the camera is carried out.

[0097] Finally, after the layout simulation of all the camera points in the semi-closed scene is completed, the final point layout scheme is output.

[0098] The above describes the basic principles, main features and advantages of the present application. It should be understood by those skilled in the art that the present application is not limited by the above examples, and the above examples and descriptions in the specification are only the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection claimed by the present application is defined by the appended claims and their equivalents.

Claims

1. A method for deploying cameras in a semi-enclosed environment, characterized in that, Includes the following steps: S100: Construct a 3D model of a semi-enclosed scene and perform area identification and division to obtain entrance / exit areas and diversion areas; S200: Select the installation locations in the entrance and exit area and simulate the deployment based on the camera's coverage area to obtain multiple first candidate deployment points with the largest coverage of the entrance and exit area; S300: For each camera at the first candidate deployment point, construct multiple vehicle bounding boxes based on the recognition error and map them from the camera coordinate system to the 3D model coordinate system; S400: Based on the mapping position error, select the first candidate deployment point within the coverage area that can identify the most vehicle bounding boxes as the best deployment point for the entrance / exit area. S500: If the perception range of the optimal placement points in the entrance / exit area cannot cover the entire semi-enclosed scene, then overlapping viewpoints will be placed from the edge to the inside of the diversion area.

2. The camera deployment method in a semi-enclosed scene as described in claim 1, characterized in that, In step S100, the obtained three-dimensional model body is preserved as a three-dimensional basic model and a cube mesh model with set specifications; The process of obtaining the first candidate deployment point through simulation in step S200 is as follows: Calculate the extrinsic parameter matrix of the camera based on its installation height, horizontal installation angle, pitch angle, and installation position on the ground. For each installable point in the entrance / exit area, construct a vector from the pixel coordinates to the camera direction in the camera coordinate system as the pixel ray; The pixel rays are mapped from the camera coordinate system to the world coordinate system and the mapping intersections with the 3D base model are solved to obtain the set of mapping intersections that all pixel rays can capture. Calculate the convex hull for the valid mapping intersections, and map the convex hull region onto the cubic mesh model to obtain the set of voxel meshes covered by the camera; The installation points with the largest number of cover voxel grids are selected as the first candidate deployment points.

3. The camera deployment method in a semi-enclosed scene as described in claim 2, characterized in that, When selecting suitable installation locations in entrance and exit areas, the following constraints apply to the camera's field of view: The field of vision must fully cover the single-lane path in the entrance and exit area; The field of vision needs to fully cover the junction of the entrance / exit area and the distribution area; The field of vision needs to fully cover the initial portion of the multi-lane area in the diversion zone.

4. The camera deployment method in a semi-enclosed scene as described in claim 2, characterized in that, Step S300 includes the following process: Construct the vehicle's bounding box based on its structural dimensions; The position of the vehicle's bounding box center point in the camera coordinate system is projected onto the image plane coordinate system, and the predicted bounding box center point of the vehicle is obtained by introducing recognition error. Based on the camera's intrinsic parameters and installation pose, the center point of the vehicle's predicted bounding box is back-projected into the 3D model coordinate system.

5. The camera deployment method in a semi-enclosed scene as described in claim 4, characterized in that, In step S400, the formula for calculating the mapped position error ε is as follows: ； Where P represents the coordinate transformation function, (u0, v0) represents the world coordinates of the vehicle bounding box center point after being back-projected from the image plane coordinate system to the ground through intrinsic and extrinsic parameters, s and t represent the pixel offsets in the horizontal and vertical directions of the image due to recognition errors, respectively, and Δx and Δy represent the theoretical pixel offset boundaries in the horizontal and vertical directions of the image due to recognition errors, respectively.

6. The camera deployment method in a semi-enclosed scene as described in claim 5, characterized in that, The selection of the optimal deployment point in step S400 includes the following process: Within the voxel grid covered by each first candidate deployment point, traverse all possible vehicle position voxels and calculate the position error ε corresponding to all vehicle position voxels. Set an error threshold and mark all vehicle position voxels with position errors ε less than the set error threshold as valid coverage voxels; The effective coverage voxels of all first candidate deployment points are counted, and the first candidate deployment point with the most effective coverage voxels that meet the premise of full coverage of the key area is selected as the best deployment point.

7. The camera deployment method in a semi-enclosed scene as described in any one of claims 2-6, characterized in that, After obtaining the optimal deployment points in the entrance and exit areas, the calculation of the sensing range of the optimal deployment points includes the following process: Camera installation location coordinates based on optimal placement points [X] A Y A ] T Calculate the center point (x) of each effective voxel grid covered by the optimal placement point. i y i The polar angle θ i , ; Based on the obtained polar angle θ i Calculate the central angle θ of the effective voxel grid region covered by the optimal placement points. center ; ， ； Based on the obtained central angle, calculate the relative angle Δθ for each effective voxel grid. i and take all relative angles Δθ i The maximum value Δθ max and minimum value Δθ min Effective horizontal perspective α of the sensing area eff Calculation; ,a eff =Δθ max -Dth min (>0); Based on the maximum distance R from the sensing boundary to the optimal deployment point max and minimum distance R min Calculate the effective sensing radius R cons =0.5(R min +R max ); The sensing range Ω of the optimal deployment point A The expression is: ； Where θ represents the sensing range Ω A The polar angle corresponding to any coordinate position (x, y) within the area, where m represents the number of effective voxel grids covered by the optimal placement point. and These represent the average angles in the y and x directions, respectively.

8. The camera deployment method in a semi-enclosed scene as described in claim 7, characterized in that, Step S500 includes the following process: In the installation locations within the diversion area, multiple second candidate deployment points are selected based on the principles of sensing intersection and maximum coverage; For each camera at a second candidate deployment point, construct multiple vehicle bounding boxes based on the recognition error and map them from the camera coordinate system to the 3D model coordinate system; The second candidate deployment point, which can identify the most vehicle bounding boxes within the coverage area and overlaps with the coverage area of ​​the previous deployment point, is selected as the optimal deployment point for deployment.

9. The camera deployment method in a semi-enclosed scene as described in claim 8, characterized in that, The second candidate deployment location must meet the following requirements: The sensing range of the deployed cameras is directed toward the centroid of the set of voxels adjacent to the sensing range of the previous optimal deployment point. The sensing range of the deployed cameras overlaps with that of the previous optimal deployment point, and the positional error of the vehicle position voxels corresponding to the overlapping area is less than the set error threshold.

10. The camera deployment method in a semi-enclosed scene as described in claim 8, characterized in that, When setting up the layout points in the diversion area, the visual penetration between each diversion sub-area in the diversion area is judged. For adjacent sub-regions that cannot be visually penetrated, cameras are independently deployed in each sub-region; For adjacent sub-regions that can be visually penetrated, determine whether the camera perception range of one sub-region can cover the other sub-region. If coverage is achieved, no additional points need to be added to the other sub-region; otherwise, additional points need to be added to the other sub-region.

Images