Multi-position camera control method and system based on cooperative scheduling optimization

Abstract

The application discloses a multi-position camera control method and system based on cooperative scheduling optimization, and the method comprises the following steps: acquiring real-time images and device positions of multiple photographic devices in a shooting area; according to the real-time images, identifying shooting object data corresponding to the shooting area based on an object recognition algorithm; determining a scheduling path corresponding to each photographic device according to the shooting object data and the device positions; and determining a cooperative scheduling scheme corresponding to at least two photographic devices based on a dynamic programming algorithm according to the scheduling paths corresponding to all the photographic devices. It can be seen that the application can realize intelligent optimization of a multi-device cooperative shooting path, improve shooting coverage efficiency and cooperativeness, and reduce the risk of shooting quality decline caused by path conflicts or insufficient coverage.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a multi-camera control method and system based on collaborative scheduling optimization. Background Technology

[0002] With the rapid growth in demand for multi-device collaborative shooting in fields such as intelligent photography and film and television production, enterprises and individuals are increasingly focusing on improving coverage efficiency and shooting quality through the collaborative scheduling of multiple cameras. A key technical issue is how to dynamically optimize multi-device collaborative scheduling schemes to avoid path conflicts and insufficient coverage. Existing technologies typically acquire real-time images and device positions from multiple cameras, using manual scheduling or simple preset path rules to control device movement, and execute shooting tasks based on fixed collaborative strategies to meet basic shooting requirements. However, existing solutions lack intelligent recognition of the shooting object, dynamic correlation analysis between object data and device positions, and the generation of collaborative scheduling schemes driven by dynamic programming algorithms. This makes it difficult to achieve intelligent optimization of multi-device shooting paths. Commonly used manual or static scheduling methods cannot adapt to complex dynamic scenes, leading to frequent path conflicts or insufficient coverage, easily causing shooting blind spots, repeated shooting, or quality degradation, thus limiting the efficiency, collaboration, and overall shooting effect of multi-device collaborative shooting. Therefore, existing technologies have shortcomings that urgently need to be addressed. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a multi-camera control method and system based on collaborative scheduling optimization, which can realize intelligent optimization of the collaborative shooting path of multiple devices, improve the shooting coverage efficiency and collaboration, and reduce the risk of shooting quality degradation caused by path conflict or insufficient coverage.

[0004] To address the aforementioned technical problems, the first aspect of this invention discloses a multi-camera control method based on cooperative scheduling optimization, the method comprising: Acquire real-time images and device locations from multiple cameras in the shooting area; Based on the real-time image, and using an object recognition algorithm, the shooting object data corresponding to the shooting area is identified; Based on the data of the object being photographed and the location of the device, a scheduling path is determined for each of the photographic devices. Based on the scheduling paths corresponding to all the aforementioned photographic devices, a collaborative scheduling scheme for at least two of the photographic devices is determined using a dynamic programming algorithm.

[0005] As an optional implementation, in the first aspect of the present invention, the step of identifying the shooting object data corresponding to the shooting area based on the real-time image and an object recognition algorithm includes: For each real-time image, the real-time image is input into the trained image segmentation algorithm model to obtain the scene object and person object corresponding to the real-time image; Based on the scene object, all the real-time images are grouped to obtain multiple image sets; Based on the image comparison algorithm, the set scene and set object corresponding to each set of images are determined; The set of scenes and set of objects corresponding to all the image sets are determined as the shooting object data corresponding to the shooting area.

[0006] As an optional implementation, in the first aspect of the invention, grouping all the real-time images based on the scene object to obtain multiple image sets includes: Calculate the scene image similarity between any two scene objects corresponding to the real-time images; Calculate the average scene image similarity between each of the real-time images and all other real-time images to obtain the corresponding first similarity parameter; Remove all real-time images whose first similarity parameter is less than a preset first parameter threshold; Based on the image similarity, all the remaining real-time images are grouped to obtain multiple image sets; the scene image similarity between any two real-time images in each image set is greater than a preset first similarity threshold, and the scene image similarity between any real-time images belonging to two different image sets is less than a preset second similarity threshold; the second similarity threshold is less than the first similarity threshold.

[0007] As an optional implementation, in the first aspect of the present invention, determining the set scene and set object corresponding to each set of images based on the image comparison algorithm includes: For each image set, the scene objects corresponding to all the real-time images in the image set are input into the trained image text description model to obtain the output set scene corresponding to the image set; Calculate the image similarity between any two real-time images in the image set corresponding to the human figures; Calculate the average similarity of the person image between each of the real-time images in the image set and all other real-time images to obtain the second similarity parameter corresponding to each real-time image; The person object corresponding to the real-time image with the highest second similarity parameter is determined, and the set object corresponding to the image set is obtained.

[0008] As an optional implementation, in the first aspect of the present invention, determining the scheduling path corresponding to each of the photographic devices based on the subject data and the device location includes: Based on the object data, establish corresponding 3D information of the object in a preset 3D scene; For each of the aforementioned photographic devices, a device model corresponding to the photographic device is established in the three-dimensional scene based on the device position corresponding to the photographic device. Based on the 3D information of the object being photographed and the device model, the scheduling path corresponding to the photographic device is calculated in the 3D scene.

[0009] As an optional implementation, in the first aspect of the present invention, the step of calculating the scheduling path corresponding to the photographic device in the three-dimensional scene based on the three-dimensional information of the photographed object and the device model includes: Based on a preset shooting range prediction model, the shooting range corresponding to the device model is predicted; In the three-dimensional scene, adjust the shooting angle and model position of the device model until the shooting range and the three-dimensional information of the shooting object meet the preset shooting coverage standard, so as to obtain the optimal shooting angle and optimal model position of the device model. Based on the current shooting angle of the camera and the optimal shooting angle, determine the shooting angle change parameters; Based on the device location and the optimal model location, determine the shooting location change parameters; The shooting angle change parameters and the shooting position change parameters are determined as the scheduling path corresponding to the photography equipment.

[0010] As an optional implementation, in the first aspect of the invention, determining a collaborative scheduling scheme for at least two of the photographic devices based on a dynamic programming algorithm, according to the scheduling paths corresponding to all the photographic devices, includes: The objective function is set to minimize the sum of the scheduling time costs corresponding to the scheduling paths of all the aforementioned photographic equipment in the scheduling scheme; the scheduling time cost is obtained by inputting the scheduling paths of the schemes into a trained time prediction model; The constraints include: There is no path conflict between the scheduling paths corresponding to any two of the aforementioned photographic devices in the scheduling scheme; Based on the objective function, the scheduling paths corresponding to at least two of the photographic devices are randomly iteratively optimized using a dynamic programming algorithm until the optimal scheduling scheme is obtained, which is then determined as the collaborative scheduling scheme corresponding to at least two of the photographic devices.

[0011] As an optional implementation, in the first aspect of the invention, the constraint conditions further include: In the scheduling scheme, the path difference between the scheduling path corresponding to any of the photographic devices is less than a preset difference threshold. In the scheduling scheme, the minimum distance between the scheduling paths corresponding to any two adjacent photographic devices is greater than a preset first distance threshold to ensure scheduling safety; the positional distance between the device locations of the adjacent photographic devices is less than a preset second distance threshold.

[0012] A second aspect of this invention discloses a multi-camera control system based on cooperative scheduling optimization, the system comprising: The acquisition module is used to acquire real-time images and device locations from multiple cameras in the shooting area; The recognition module is used to identify the shooting object data corresponding to the shooting area based on the real-time image and an object recognition algorithm. The determination module is used to determine the scheduling path corresponding to each of the photography devices based on the shooting object data and the device location; The scheduling module is used to determine a collaborative scheduling scheme for at least two of the photographic devices based on a dynamic programming algorithm, according to the scheduling paths corresponding to all the photographic devices.

[0013] As an optional implementation, in a second aspect of the present invention, the specific method by which the recognition module identifies the shooting object data corresponding to the shooting area based on the real-time image and an object recognition algorithm includes: For each real-time image, the real-time image is input into the trained image segmentation algorithm model to obtain the scene object and person object corresponding to the real-time image; Based on the scene object, all the real-time images are grouped to obtain multiple image sets; Based on the image comparison algorithm, the set scene and set object corresponding to each set of images are determined; The set of scenes and set of objects corresponding to all the image sets are determined as the shooting object data corresponding to the shooting area.

[0014] As an optional implementation, in a second aspect of the invention, the specific method by which the recognition module groups all the real-time images based on the scene object to obtain multiple image sets includes: Calculate the scene image similarity between any two scene objects corresponding to the real-time images; Calculate the average scene image similarity between each of the real-time images and all other real-time images to obtain the corresponding first similarity parameter; Remove all real-time images whose first similarity parameter is less than a preset first parameter threshold; Based on the image similarity, all the remaining real-time images are grouped to obtain multiple image sets; the scene image similarity between any two real-time images in each image set is greater than a preset first similarity threshold, and the scene image similarity between any real-time images belonging to two different image sets is less than a preset second similarity threshold; the second similarity threshold is less than the first similarity threshold.

[0015] As an optional implementation, in a second aspect of the invention, the recognition module determines the specific method by which it determines the set scene and set object corresponding to each set of images based on an image comparison algorithm, including: For each image set, the scene objects corresponding to all the real-time images in the image set are input into the trained image text description model to obtain the output set scene corresponding to the image set; Calculate the image similarity between any two real-time images in the image set corresponding to the human figures; Calculate the average similarity of the person image between each of the real-time images in the image set and all other real-time images to obtain the second similarity parameter corresponding to each real-time image; The person object corresponding to the real-time image with the highest second similarity parameter is determined, and the set object corresponding to the image set is obtained.

[0016] As an optional implementation, in a second aspect of the invention, the determining module determines the specific method of the scheduling path corresponding to each of the photographic devices based on the subject data and the device location, including: Based on the object data, establish corresponding 3D information of the object in a preset 3D scene; For each of the aforementioned photographic devices, a device model corresponding to the photographic device is established in the three-dimensional scene based on the device position corresponding to the photographic device. Based on the 3D information of the object being photographed and the device model, the scheduling path corresponding to the photographic device is calculated in the 3D scene.

[0017] As an optional implementation, in a second aspect of the invention, the determining module calculates the specific method of the scheduling path corresponding to the photographic device in the three-dimensional scene based on the three-dimensional information of the object being photographed and the device model, including: Based on a preset shooting range prediction model, the shooting range corresponding to the device model is predicted; In the three-dimensional scene, adjust the shooting angle and model position of the device model until the shooting range and the three-dimensional information of the shooting object meet the preset shooting coverage standard, so as to obtain the optimal shooting angle and optimal model position of the device model. Based on the current shooting angle of the camera and the optimal shooting angle, determine the shooting angle change parameters; Based on the device location and the optimal model location, determine the shooting location change parameters; The shooting angle change parameters and the shooting position change parameters are determined as the scheduling path corresponding to the photography equipment.

[0018] As an optional implementation, in a second aspect of the invention, the scheduling module determines, based on a dynamic programming algorithm, the specific method of a collaborative scheduling scheme for at least two of the photographic devices according to the scheduling paths corresponding to all the photographic devices, including: The objective function is set to minimize the sum of the scheduling time costs corresponding to the scheduling paths of all the aforementioned photographic equipment in the scheduling scheme; the scheduling time cost is obtained by inputting the scheduling paths of the schemes into a trained time prediction model; The constraints include: There is no path conflict between the scheduling paths corresponding to any two of the aforementioned photographic devices in the scheduling scheme; Based on the objective function, the scheduling paths corresponding to at least two of the photographic devices are randomly iteratively optimized using a dynamic programming algorithm until the optimal scheduling scheme is obtained, which is then determined as the collaborative scheduling scheme corresponding to at least two of the photographic devices.

[0019] As an optional implementation, in a second aspect of the invention, the constraints further include: In the scheduling scheme, the path difference between the scheduling path corresponding to any of the photographic devices is less than a preset difference threshold. In the scheduling scheme, the minimum distance between the scheduling paths corresponding to any two adjacent photographic devices is greater than a preset first distance threshold to ensure scheduling safety; the positional distance between the device locations of the adjacent photographic devices is less than a preset second distance threshold.

[0020] A third aspect of this invention discloses another multi-camera control system based on cooperative scheduling optimization, the system comprising: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute some or all of the steps in the multi-camera control method based on cooperative scheduling optimization disclosed in the first aspect of the present invention.

[0021] The fourth aspect of the present invention discloses a computer storage medium storing computer instructions, which, when invoked, are used to execute some or all of the steps in the multi-camera control method based on cooperative scheduling optimization disclosed in the first aspect of the present invention.

[0022] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: This invention acquires real-time images and device locations of multiple cameras in a shooting area, identifies shooting object data based on an object recognition algorithm, determines a scheduling path based on the object data and device location, and generates a collaborative scheduling scheme for at least two cameras based on a dynamic programming algorithm. This enables intelligent optimization of multi-device collaborative shooting paths, improves shooting coverage efficiency and collaboration, and reduces the risk of shooting quality degradation due to path conflicts or insufficient coverage. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a multi-camera control method based on collaborative scheduling optimization disclosed in an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram of a multi-camera control system based on collaborative scheduling optimization disclosed in an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram of another multi-camera control system based on collaborative scheduling optimization disclosed in an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0029] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0030] This invention discloses a multi-camera control method and system based on cooperative scheduling optimization. By acquiring real-time images and device positions of multiple cameras in the shooting area, it identifies the shooting object data based on an object recognition algorithm, determines the scheduling path based on the object data and device positions, and generates a cooperative scheduling scheme for at least two cameras based on a dynamic programming algorithm. This enables intelligent optimization of the multi-device cooperative shooting path, improves shooting coverage efficiency and coordination, and reduces the risk of shooting quality degradation due to path conflicts or insufficient coverage. Detailed explanations follow.

[0031] Example 1 Please see Figure 1 , Figure 1 This is a flowchart illustrating a multi-camera control method based on collaborative scheduling optimization disclosed in an embodiment of the present invention. Figure 1 The described multi-camera control method based on collaborative scheduling optimization can be applied to data processing systems / data processing equipment / data processing servers (wherein, the server includes a local processing server or a cloud processing server). For example... Figure 1 As shown, the multi-camera control method based on cooperative scheduling optimization can include the following operations: 101. Obtain real-time images and device locations of multiple cameras in the shooting area.

[0032] Optionally, the photography equipment can be a 4K high-definition PTZ camera, an infrared thermal imaging camera, a PTZ camera with gimbal control function, an industrial drone equipped with a camera, or a smart mobile terminal; this invention does not limit the scope of the equipment.

[0033] Optionally, the real-time image can be an RGB color image, a YUV format video stream, a depth map, or an ultra-high-definition video stream compressed with H.265 encoding; the present invention does not impose any limitations on this.

[0034] Optionally, the location of the device can be latitude and longitude coordinates obtained by the Global Positioning System (GPS), indoor relative coordinates obtained by ultra-wideband (UWB) positioning technology, real-time pose data calculated based on the SLAM algorithm, or static coordinate points preset on architectural drawings. This invention does not impose any limitations on these parameters.

[0035] 102. Based on real-time images and an object recognition algorithm, identify the shooting object data corresponding to the shooting area. Optionally, the object recognition algorithm can be a deep learning-based YOLOv8, Faster R-CNN, SSD object detection algorithm, or a DETR recognition model based on the Transformer architecture; this invention does not limit the specific algorithm.

[0036] Optionally, the captured object data can be structured JSON data containing target ID, center point coordinates, bounding box size, confidence score, and motion vector information; this invention does not impose any limitations on this.

[0037] 103. Based on the data of the shooting object and the location of the equipment, determine the scheduling path corresponding to each shooting device. Optionally, the scheduling path can be a series of discrete coordinate waypoints in three-dimensional space, a sequence of gimbal rotation angles represented by quaternions, or a collision-free motion trajectory planned based on the A algorithm or RRT algorithm. This invention does not limit the scope of the path.

[0038] 104. Based on the scheduling paths corresponding to all photography equipment, determine the collaborative scheduling scheme for at least two photography equipment using a dynamic programming algorithm.

[0039] Optionally, the dynamic programming algorithm can be a path selection model based on Bellman-Ford logic, the Viterbi Algorithm, or a heuristic search strategy with a state transition probability matrix; this invention does not limit the specific algorithm.

[0040] Optionally, the collaborative scheduling scheme can be a multi-device regional coverage instruction, a master-slave device alternating tracking task list, or a distributed recording instruction set triggered by time synchronization; this invention does not limit the specific implementation.

[0041] As can be seen, the above-described embodiments of the invention acquire real-time images and device locations of multiple photography devices in the shooting area, identify shooting object data based on object recognition algorithms, determine scheduling paths based on object data and device locations, and generate collaborative scheduling schemes for at least two photography devices based on dynamic programming algorithms. This enables intelligent optimization of multi-device collaborative shooting paths, improves shooting coverage efficiency and collaboration, and reduces the risk of shooting quality degradation due to path conflicts or insufficient coverage.

[0042] As an optional embodiment, the step above, identifying the shooting object data corresponding to the shooting area based on the real-time image and an object recognition algorithm, includes: For each real-time image, the real-time image is input into the trained image segmentation algorithm model to obtain the scene objects and human objects corresponding to the real-time image; Based on scene objects, all real-time images are grouped to obtain multiple image sets; Based on image comparison algorithms, the set scene and set object corresponding to each image set are determined; The scene and objects corresponding to all image sets are identified as the shooting object data corresponding to the shooting area.

[0043] Optionally, the image segmentation algorithm model can be Mask R-CNN, DeepLabV3+, U-Net, or a semantic segmentation network based on SwinTransformer; this invention does not limit the specific model.

[0044] Optionally, the scene object can be a building, green plants, sky, road signs, or a static photographic obstacle model; this invention does not impose any limitations.

[0045] Optionally, the person object can be a skeleton model defined by human keypoints, a person entity characterized by face recognition, or a pedestrian re-identification feature vector based on ReID technology; this invention does not impose any limitations.

[0046] As can be seen, through the above optional embodiments, by inputting each real-time image into the trained image segmentation algorithm model to obtain scene objects and human objects, grouping the images based on scene objects to obtain an image set, and determining the set scene and set objects as the shooting object data, accurate shooting object recognition based on segmentation and clustering is achieved, improving the accuracy and completeness of object data extraction, and reducing the risk of object recognition deviation caused by image clutter.

[0047] As an optional embodiment, the above step of grouping all real-time images based on scene objects to obtain multiple image sets includes: Calculate the scene image similarity between scene objects corresponding to any two real-time images; Calculate the average scene image similarity between each real-time image and all other real-time images to obtain the corresponding first similarity parameter; Remove all real-time images whose first similarity parameter is less than a preset first parameter threshold; The remaining real-time images are grouped based on image similarity to obtain multiple image sets.

[0048] Optionally, the scene image similarity between any two real-time images in each image set is greater than a preset first similarity threshold, and the scene image similarity between any real-time images belonging to two different image sets is less than a preset second similarity threshold; the second similarity threshold is less than the first similarity threshold.

[0049] Optionally, the scene image similarity can be based on the feature vector distance calculated by cosine similarity, the color matching degree based on the histogram intersection method, or the image integrity assessment based on the structural similarity index SSIM. This invention does not limit the assessment.

[0050] As can be seen, through the above optional embodiments, by calculating the scene image similarity between any two real-time image scene objects, calculating the average similarity between each image and all other images to filter images, and grouping the remaining images based on similarity to obtain an image set, accurate image clustering based on scene similarity is achieved, improving the cohesion and representativeness of the image set, and reducing the risk of grouping errors caused by insufficient similarity calculation.

[0051] As an optional embodiment, the step above, determining the set scene and set object corresponding to each image set based on the image comparison algorithm, includes: For each image set, the scene objects corresponding to all real-time images in the image set are input into the trained image text description model to obtain the output set scene corresponding to the image set; Calculate the image similarity between any two real-time images of people in the image set; Calculate the average similarity of the human images between each real-time image in the image set and all other real-time images to obtain the second similarity parameter corresponding to each real-time image; The person object corresponding to the real-time image with the highest second similarity parameter is identified, and the set object corresponding to the image set is obtained.

[0052] Optionally, the image text description model can be a CLIP model, a BLIP-2 model, a Show and Tell model, or a multimodal understanding model based on the GPT-4V interface; this invention does not impose any limitations.

[0053] Optionally, the set of scenarios can be text phrases describing environmental semantics (such as "indoor convention center" or "outdoor lawn wedding") or corresponding scenario classification tag codes, which are not limited in this invention.

[0054] As can be seen, through the above optional embodiments, by inputting the scene objects of all real-time images in the image set into the image text description model to obtain the set scene, and calculating the similarity of the human objects to select the human objects with the highest similarity as the set objects, the accurate determination of the set objects based on the fusion of text description and similarity is achieved, which improves the semantic accuracy of the captured object data and reduces the risk of set feature deviation caused by the mixing of human objects.

[0055] As an optional embodiment, the step of determining the scheduling path corresponding to each camera device based on the subject data and device location, as described above, includes: Based on the data of the photographed object, establish the corresponding 3D information of the photographed object in the preset 3D scene; For each camera device, a model of that camera device is built in the 3D scene based on its location. Based on the 3D information of the subject and the device model, the scheduling path corresponding to the camera device is calculated in the 3D scene.

[0056] Optionally, the three-dimensional scene can be an OSGB format real-world three-dimensional model generated by oblique photography, a Voxel occupancy raster map based on point cloud data, or a digital building model constructed based on BIM technology; the present invention does not limit this.

[0057] Optionally, the three-dimensional information of the photographed object can be a three-dimensional point cloud cluster, bounding volume model (OBB / AABB), or mesh data with volume properties in the world coordinate system. This invention does not limit the scope of the invention.

[0058] Optionally, the device model can be a frustum model containing sensor field-of-view parameters, a gimbal physical model with three degrees of freedom, or a coverage radius model simplified to a sphere-centered coordinate system; the present invention does not limit this.

[0059] As can be seen, through the above optional embodiments, by establishing three-dimensional information of the shooting object in a preset three-dimensional scene based on the shooting object data, and establishing a device model for each photography device based on the device position, the scheduling path is calculated in the three-dimensional scene, realizing accurate device scheduling path planning based on three-dimensional spatial modeling, improving the three-dimensionality and rationality of path generation, and reducing the risk of path deviation caused by planar calculation.

[0060] As an optional embodiment, the step described above, calculating the scheduling path corresponding to the photographic device in the 3D scene based on the 3D information of the object being photographed and the device model, includes: Based on a preset shooting range prediction model, predict the shooting range corresponding to the device model; Adjust the shooting angle and position of the device model in the 3D scene until the shooting range and the 3D information of the shooting object meet the preset shooting coverage standard, and obtain the optimal shooting angle and optimal model position of the device model. Based on the current and optimal shooting angles of the camera equipment, determine the shooting angle variation parameters; Based on the device location and the optimal model location, determine the shooting location change parameters; The shooting angle change parameters and shooting position change parameters are used to determine the scheduling path corresponding to the camera equipment.

[0061] Optionally, the imaging range prediction model can be a projection model based on optical geometric parameters, a line-of-sight occlusion analysis model based on ray tracing algorithm, or an effective imaging area prediction network based on neural network regression; the present invention does not limit this.

[0062] Optionally, the shooting range can be a fan-shaped area defined by the field of view (FOV), a three-dimensional visual object, or the effective projection area of ​​the image sensor on the target plane; this invention does not limit this range.

[0063] Optionally, the shooting coverage standard can be an overlap rate IoU > 0.5, a key point visibility greater than 80%, or the target object occupying a pixel ratio of 30% - 60% in the image. This invention does not impose any limitations.

[0064] Optionally, the optimal shooting angle can be a combination of pitch and yaw angles that maximizes the feature point extraction effect; this invention does not limit this.

[0065] As can be seen, through the above optional embodiments, the shooting range of the device model is predicted by the shooting range prediction model, the angle and position of the device model are adjusted until the shooting coverage standard is met, the optimal angle and position are determined and the scheduling path is generated, so as to realize the accurate shooting path calculation based on the coverage standard optimization, improve the adaptability and efficiency of the path to the shooting needs, and reduce the risk of scheduling failure due to insufficient coverage.

[0066] As an optional embodiment, the step described above, determining a collaborative scheduling scheme for at least two camera devices based on a dynamic programming algorithm, according to the scheduling paths corresponding to all camera devices, includes: The objective function is set to minimize the sum of the scheduling time costs corresponding to the scheduling paths of all photographic equipment in the scheduling scheme. The constraints include: There are no path conflicts between the scheduling paths corresponding to any two photography devices in the scheduling scheme; Based on the objective function, the scheduling paths for at least two camera devices are randomly iteratively optimized using a dynamic programming algorithm until the optimal scheduling scheme is obtained, which is then determined as the collaborative scheduling scheme for at least two camera devices.

[0067] Optionally, the objective function may include solving the shortest time path problem under multiple constraints, minimizing power consumption, or maximizing a comprehensive evaluation index of coverage quality; this invention does not impose any limitations on this.

[0068] Optionally, the scheduling time cost may include the sum of motion time calculated from motor speed, the ratio of path length to movement speed, and algorithm processing delay; this invention does not limit this.

[0069] Optionally, the scheduling time cost is obtained by inputting the scheduling path of the scheme into a trained time prediction model.

[0070] Optionally, the time prediction model can be an ensemble learning algorithm model, an LSTM-based time series regression model, or a time estimation network based on physics engine dynamics simulation; this invention does not limit the model.

[0071] As can be seen, through the above optional embodiments, by setting the minimum total scheduling time cost as the objective function, and combining the constraints of path conflict, a collaborative scheduling scheme is obtained based on dynamic programming iterative optimization. This achieves optimal collaborative path planning under multi-objective constraints, improves the overall efficiency and security of multi-device scheduling schemes, and reduces the risk of scheduling conflicts caused by insufficient consideration of constraints.

[0072] As an optional embodiment, the constraints in the above steps further include: In the scheduling scheme, the path difference between the scheduling path corresponding to any camera device and the corresponding scheduling path is less than the preset difference threshold. In the scheduling scheme, the minimum distance between the scheduling paths corresponding to any two adjacent camera devices is greater than a preset first distance threshold to ensure scheduling safety; the positional distance between adjacent camera devices is less than a preset second distance threshold.

[0073] Optionally, the path difference can be the Hausdorff distance between the two curves, the dynamic time warping (DTW) distance, or the root mean square error based on RMSE; this invention does not limit the specific method.

[0074] Optionally, the first distance threshold can be 0.5 meters, 2 meters, or a safe obstacle avoidance radius determined based on the physical dimensions of the device; the present invention does not impose any limitation on this.

[0075] Optionally, the second distance threshold can be based on the effective connection radius of the communication module (such as a Wi-Fi coverage range of 100 meters) or the overlapping baseline length required for collaborative visual feature extraction; this invention does not limit this.

[0076] As can be seen, the above optional embodiments further define the content of the constraints, introduce path difference degree and distance constraints between devices, which can help realize the optimal collaborative path planning under multiple constraints and improve the overall efficiency and security of multi-device scheduling schemes.

[0077] Example 2 Please see Figure 2 , Figure 2 This is a schematic diagram of a multi-camera control system based on collaborative scheduling optimization disclosed in an embodiment of the present invention. Figure 2 The described multi-camera camera control system based on collaborative scheduling optimization can be applied to data processing systems / data processing equipment / data processing servers (wherein, the server includes a local processing server or a cloud processing server). For example... Figure 2 As shown, the multi-camera control system based on collaborative scheduling optimization may include: The acquisition module 201 is used to acquire real-time images and device locations of multiple photography devices in the shooting area.

[0078] The recognition module 202 is used to identify the shooting object data corresponding to the shooting area based on the real-time image and the object recognition algorithm. The determination module 203 is used to determine the scheduling path corresponding to each camera device based on the shooting object data and the device location. The scheduling module 204 is used to determine a collaborative scheduling scheme for at least two cameras based on a dynamic programming algorithm, according to the scheduling paths corresponding to all cameras.

[0079] As can be seen, the above-described embodiments of the invention acquire real-time images and device locations of multiple photography devices in the shooting area, identify shooting object data based on object recognition algorithms, determine scheduling paths based on object data and device locations, and generate collaborative scheduling schemes for at least two photography devices based on dynamic programming algorithms. This enables intelligent optimization of multi-device collaborative shooting paths, improves shooting coverage efficiency and collaboration, and reduces the risk of shooting quality degradation due to path conflicts or insufficient coverage.

[0080] As an optional embodiment, the specific method by which the recognition module identifies the shooting object data corresponding to the shooting area based on the real-time image and an object recognition algorithm includes: For each real-time image, the real-time image is input into the trained image segmentation algorithm model to obtain the scene objects and human objects corresponding to the real-time image; Based on scene objects, all real-time images are grouped to obtain multiple image sets; Based on image comparison algorithms, the set scene and set object corresponding to each image set are determined; The scene and objects corresponding to all image sets are identified as the shooting object data corresponding to the shooting area.

[0081] As can be seen, through the above optional embodiments, by inputting each real-time image into the trained image segmentation algorithm model to obtain scene objects and human objects, grouping the images based on scene objects to obtain an image set, and determining the set scene and set objects as the shooting object data, accurate shooting object recognition based on segmentation and clustering is achieved, improving the accuracy and completeness of object data extraction, and reducing the risk of object recognition deviation caused by image clutter.

[0082] As an optional embodiment, the recognition module groups all real-time images based on scene objects to obtain multiple image sets in the following specific ways: Calculate the scene image similarity between scene objects corresponding to any two real-time images; Calculate the average scene image similarity between each real-time image and all other real-time images to obtain the corresponding first similarity parameter; Remove all real-time images whose first similarity parameter is less than a preset first parameter threshold; Based on image similarity, all remaining real-time images are grouped to obtain multiple image sets; optionally, the scene image similarity between any two real-time images in each image set is greater than a preset first similarity threshold, and the scene image similarity between any real-time images belonging to two different image sets is less than a preset second similarity threshold; the second similarity threshold is less than the first similarity threshold.

[0083] As can be seen, through the above optional embodiments, by calculating the scene image similarity between any two real-time image scene objects, calculating the average similarity between each image and all other images to filter images, and grouping the remaining images based on similarity to obtain an image set, accurate image clustering based on scene similarity is achieved, improving the cohesion and representativeness of the image set, and reducing the risk of grouping errors caused by insufficient similarity calculation.

[0084] As an optional embodiment, the recognition module determines the specific method of the set scene and set object corresponding to each image set based on the image comparison algorithm, including: For each image set, the scene objects corresponding to all real-time images in the image set are input into the trained image text description model to obtain the output set scene corresponding to the image set; Calculate the image similarity between any two real-time images of people in the image set; Calculate the average similarity of the human images between each real-time image in the image set and all other real-time images to obtain the second similarity parameter corresponding to each real-time image; The person object corresponding to the real-time image with the highest second similarity parameter is identified, and the set object corresponding to the image set is obtained.

[0085] As can be seen, through the above optional embodiments, by inputting the scene objects of all real-time images in the image set into the image text description model to obtain the set scene, and calculating the similarity of the human objects to select the human objects with the highest similarity as the set objects, the accurate determination of the set objects based on the fusion of text description and similarity is achieved, which improves the semantic accuracy of the captured object data and reduces the risk of set feature deviation caused by the mixing of human objects.

[0086] As an optional embodiment, the determining module determines the specific method of the scheduling path corresponding to each camera device based on the shooting object data and device location, including: Based on the data of the photographed object, establish the corresponding 3D information of the photographed object in the preset 3D scene; For each camera device, a model of that camera device is built in the 3D scene based on its location. Based on the 3D information of the subject and the device model, the scheduling path corresponding to the camera device is calculated in the 3D scene.

[0087] As can be seen, through the above optional embodiments, by establishing three-dimensional information of the shooting object in a preset three-dimensional scene based on the shooting object data, and establishing a device model for each photography device based on the device position, the scheduling path is calculated in the three-dimensional scene, realizing accurate device scheduling path planning based on three-dimensional spatial modeling, improving the three-dimensionality and rationality of path generation, and reducing the risk of path deviation caused by planar calculation.

[0088] As an optional embodiment, the determining module calculates the specific method of the scheduling path corresponding to the photographic equipment in the 3D scene based on the 3D information of the photographed object and the equipment model, including: Based on a preset shooting range prediction model, predict the shooting range corresponding to the device model; Adjust the shooting angle and position of the device model in the 3D scene until the shooting range and the 3D information of the shooting object meet the preset shooting coverage standard, and obtain the optimal shooting angle and optimal model position of the device model. Based on the current and optimal shooting angles of the camera equipment, determine the shooting angle variation parameters; Based on the device location and the optimal model location, determine the shooting location change parameters; The shooting angle change parameters and shooting position change parameters are used to determine the scheduling path corresponding to the camera equipment.

[0089] As can be seen, through the above optional embodiments, the shooting range of the device model is predicted by the shooting range prediction model, the angle and position of the device model are adjusted until the shooting coverage standard is met, the optimal angle and position are determined and the scheduling path is generated, so as to realize the accurate shooting path calculation based on the coverage standard optimization, improve the adaptability and efficiency of the path to the shooting needs, and reduce the risk of scheduling failure due to insufficient coverage.

[0090] As an optional embodiment, the scheduling module determines the specific method of the collaborative scheduling scheme for at least two camera devices based on the scheduling paths corresponding to all camera devices and using a dynamic programming algorithm, including: The objective function is set to minimize the sum of scheduling time costs corresponding to the scheduling paths of all photographic equipment in the scheduling scheme; optionally, the scheduling time cost is obtained by inputting the scheduling paths of the scheme into a trained time prediction model. The constraints include: There are no path conflicts between the scheduling paths corresponding to any two photography devices in the scheduling scheme; Based on the objective function, the scheduling paths for at least two camera devices are randomly iteratively optimized using a dynamic programming algorithm until the optimal scheduling scheme is obtained, which is then determined as the collaborative scheduling scheme for at least two camera devices.

[0091] As can be seen, through the above optional embodiments, by setting the minimum total scheduling time cost as the objective function, and combining the constraints of path conflict, a collaborative scheduling scheme is obtained based on dynamic programming iterative optimization. This achieves optimal collaborative path planning under multi-objective constraints, improves the overall efficiency and security of multi-device scheduling schemes, and reduces the risk of scheduling conflicts caused by insufficient consideration of constraints.

[0092] As an optional embodiment, the constraints also include: In the scheduling scheme, the path difference between the scheduling path corresponding to any camera device and the corresponding scheduling path is less than the preset difference threshold. In the scheduling scheme, the minimum distance between the scheduling paths corresponding to any two adjacent camera devices is greater than a preset first distance threshold to ensure scheduling safety; the positional distance between adjacent camera devices is less than a preset second distance threshold.

[0093] As can be seen, the above optional embodiments further define the content of the constraints, introduce path difference degree and distance constraints between devices, which can help realize the optimal collaborative path planning under multiple constraints and improve the overall efficiency and security of multi-device scheduling schemes.

[0094] Example 3 Please see Figure 3 , Figure 3 This is another multi-camera control system based on collaborative scheduling optimization disclosed in the embodiments of the present invention. Figure 3 The described multi-camera camera control system based on collaborative scheduling optimization is applied in data processing systems / data processing equipment / data processing servers (wherein, the server includes a local processing server or a cloud processing server). For example... Figure 3 As shown, the multi-camera control system based on collaborative scheduling optimization may include: Memory 301 storing executable program code; Processor 302 coupled to memory 301; The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the multi-camera control method based on cooperative scheduling optimization described in Embodiment 1.

[0095] Example 4 This invention discloses a computer read storage medium that stores a computer program for electronic data interchange, wherein the computer program causes a computer to execute the steps of the multi-camera control method based on cooperative scheduling optimization described in Embodiment 1.

[0096] Example 5 This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps of the multi-camera control method based on cooperative scheduling optimization described in Embodiment 1.

[0097] The foregoing has described specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than those shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily have to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0098] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0099] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0100] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0101] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0102] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0103] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0104] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0105] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0106] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0107] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0108] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0109] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0110] Finally, it should be noted that the multi-camera control method and system based on collaborative scheduling optimization disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-camera control method based on cooperative scheduling optimization, characterized in that, The method includes: Acquire real-time images and device locations from multiple cameras in the shooting area; Based on the real-time image, and using an object recognition algorithm, the shooting object data corresponding to the shooting area is identified; Based on the data of the object being photographed and the location of the device, a scheduling path is determined for each of the photographic devices. Based on the scheduling paths corresponding to all the aforementioned photographic devices, a collaborative scheduling scheme for at least two of the photographic devices is determined using a dynamic programming algorithm.

2. The multi-camera control method based on cooperative scheduling optimization according to claim 1, characterized in that, The step of identifying the shooting object data corresponding to the shooting area based on the real-time image and an object recognition algorithm includes: For each real-time image, the real-time image is input into the trained image segmentation algorithm model to obtain the scene object and person object corresponding to the real-time image; Based on the scene object, all the real-time images are grouped to obtain multiple image sets; Based on the image comparison algorithm, the set scene and set object corresponding to each set of images are determined; The set of scenes and set of objects corresponding to all the image sets are determined as the shooting object data corresponding to the shooting area.

3. The multi-camera control method based on cooperative scheduling optimization according to claim 2, characterized in that, Based on the scene object, all the real-time images are grouped to obtain multiple image sets, including: Calculate the scene image similarity between any two scene objects corresponding to the real-time images; Calculate the average scene image similarity between each of the real-time images and all other real-time images to obtain the corresponding first similarity parameter; Remove all real-time images whose first similarity parameter is less than a preset first parameter threshold; Based on the image similarity, all the remaining real-time images are grouped to obtain multiple image sets; the scene image similarity between any two real-time images in each image set is greater than a preset first similarity threshold, and the scene image similarity between any real-time images belonging to two different image sets is less than a preset second similarity threshold; the second similarity threshold is less than the first similarity threshold.

4. The multi-camera control method based on cooperative scheduling optimization according to claim 2, characterized in that, The image comparison algorithm determines the set scene and set object corresponding to each set of images, including: For each image set, the scene objects corresponding to all the real-time images in the image set are input into the trained image text description model to obtain the output set scene corresponding to the image set; Calculate the image similarity between any two real-time images in the image set corresponding to the human figures; Calculate the average similarity of the person image between each of the real-time images in the image set and all other real-time images to obtain the second similarity parameter corresponding to each real-time image; The person object corresponding to the real-time image with the highest second similarity parameter is determined, and the set object corresponding to the image set is obtained.

5. The multi-camera control method based on cooperative scheduling optimization according to claim 1, characterized in that, The step of determining the scheduling path corresponding to each of the photographic devices based on the subject data and the device location includes: Based on the object data, establish corresponding 3D information of the object in a preset 3D scene; For each of the aforementioned photographic devices, a device model corresponding to the photographic device is established in the three-dimensional scene based on the device position corresponding to the photographic device. Based on the 3D information of the object being photographed and the device model, the scheduling path corresponding to the photographic device is calculated in the 3D scene.

6. The multi-camera control method based on cooperative scheduling optimization according to claim 5, characterized in that, The step of calculating the scheduling path corresponding to the photography device in the three-dimensional scene based on the three-dimensional information of the object being photographed and the device model includes: Based on a preset shooting range prediction model, the shooting range corresponding to the device model is predicted; In the three-dimensional scene, adjust the shooting angle and model position of the device model until the shooting range and the three-dimensional information of the shooting object meet the preset shooting coverage standard, so as to obtain the optimal shooting angle and optimal model position of the device model. Based on the current shooting angle of the camera and the optimal shooting angle, determine the shooting angle change parameters; Based on the device location and the optimal model location, determine the shooting location change parameters; The shooting angle change parameters and the shooting position change parameters are determined as the scheduling path corresponding to the photography equipment.

7. The multi-camera control method based on cooperative scheduling optimization according to claim 1, characterized in that, The step of determining a collaborative scheduling scheme for at least two of the photographic devices based on the scheduling paths corresponding to all the photographic devices and using a dynamic programming algorithm includes: The objective function is set to minimize the sum of the scheduling time costs corresponding to the scheduling paths of all the aforementioned photographic equipment in the scheduling scheme; the scheduling time cost is obtained by inputting the scheduling paths of the schemes into a trained time prediction model; The constraints include: There is no path conflict between the scheduling paths corresponding to any two of the aforementioned photographic devices in the scheduling scheme; Based on the objective function, the scheduling paths corresponding to at least two of the photographic devices are randomly iteratively optimized using a dynamic programming algorithm until the optimal scheduling scheme is obtained, which is then determined as the collaborative scheduling scheme corresponding to at least two of the photographic devices.

8. The multi-camera control method based on cooperative scheduling optimization according to claim 7, characterized in that, The constraints also include: In the scheduling scheme, the path difference between the scheduling path corresponding to any of the photographic devices is less than a preset difference threshold. In the scheduling scheme, the minimum distance between the scheduling paths corresponding to any two adjacent photographic devices is greater than a preset first distance threshold to ensure scheduling safety; the positional distance between the device locations of the adjacent photographic devices is less than a preset second distance threshold.

9. A multi-camera control system based on collaborative scheduling optimization, characterized in that, The system includes: The acquisition module is used to acquire real-time images and device locations from multiple cameras in the shooting area; The recognition module is used to identify the shooting object data corresponding to the shooting area based on the real-time image and an object recognition algorithm. The determination module is used to determine the scheduling path corresponding to each of the photography devices based on the shooting object data and the device location; The scheduling module is used to determine a collaborative scheduling scheme for at least two of the photographic devices based on a dynamic programming algorithm, according to the scheduling paths corresponding to all the photographic devices.

10. A multi-camera control system based on collaborative scheduling optimization, characterized in that, The system includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the multi-camera control method based on cooperative scheduling optimization as described in any one of claims 1-8.

Images