Illuminating light automatic following method, device and equipment and storage medium

Abstract

The invention provides an illumination light automatic following method, device and equipment and a storage medium, and the method comprises the steps: erecting a stereoscopic vision system in a to-be-shot region, and carrying out the stereoscopic calibration of the stereoscopic vision system, and obtaining system calibration parameters; acquiring a first image set of the to-be-shot area through the stereoscopic vision system, and performing stereoscopic correction on the first image set based on the system calibration parameters to obtain a second image set; detecting a target object in the second image set, and marking position coordinates of the target object to obtain a third image set; extracting motion features of the target based on the third image set, and generating motion track data of the target object; converting the motion trail data into light control parameters; and controlling light to automatically follow the target object based on the light control parameters.

Description

Technical Field

[0001] This application relates to the field of stage lighting technology, and in particular to a method, apparatus, device and storage medium for automatic lighting following. Background Technology

[0002] Currently, stage follow spot systems primarily rely on manual operation. However, with the increasing complexity of modern stage performances, this traditional method has revealed numerous limitations: operators need to set up complex follow spot platforms and high-altitude work equipment above the stage, posing significant safety hazards. In large-scale performances, manual follow spotting struggles to simultaneously track multiple fast-moving performers, frequently resulting in missed or incorrect tracking. Especially in special effects scenes requiring precise coordination, the delays and instabilities of manual operation can easily disrupt the overall artistic effect. Furthermore, constructing complex manual follow spot systems increases performance costs and limits the scope for innovative stage design. Summary of the Invention

[0003] This application provides a method, device, equipment, and storage medium for automatic lighting tracking. By organically combining a stereoscopic vision system with automatic control technology, it realizes the intelligent and automated operation of stage lighting tracking, effectively solving the safety hazards and insufficient precision problems of manual operation.

[0004] In a first aspect, embodiments of this application provide a method for automatic lighting follow-up, the method comprising: A stereo vision system is set up in the area to be photographed, and the stereo vision system is stereo calibrated to obtain the system calibration parameters; The first image set of the area to be photographed is obtained through the stereo vision system, and the first image set is stereo corrected based on the system calibration parameters to obtain the second image set; The target object is detected in the second image set, and the position coordinates of the target object are marked to obtain the third image set; Based on the third image set, the motion features of the target are extracted to generate the motion trajectory data of the target object; The motion trajectory data is converted into lighting control parameters; Based on the aforementioned lighting control parameters, the lighting is controlled to automatically follow the target object.

[0005] Secondly, embodiments of this application provide an automatic lighting follower device, which includes: The parameter calibration module is used to set up a stereo vision system in the area to be photographed and to perform stereo calibration on the stereo vision system to obtain system calibration parameters. The image acquisition module is used to acquire a first image set of the area to be photographed through the stereo vision system, and to perform stereo correction on the first image set based on the system calibration parameters to obtain a second image set; A location calibration module is used to detect target objects in the second image set and mark the location coordinates of the target objects to obtain a third image set; The trajectory synthesis module is used to extract the motion features of the target based on the third image set and generate the motion trajectory data of the target object; The parameter conversion module is used to convert the motion trajectory data into lighting control parameters; The instruction generation module is used to control the illumination direction and illumination characteristics of the lighting device in real time through the motor drive system and the light source adjustment system based on the lighting control instruction set, so as to realize the automatic tracking of the light to the target.

[0006] Thirdly, embodiments of this application provide a lighting device, which includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, when executing the computer program, implement the automatic lighting following method as described in any of the embodiments of this application.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the automatic lighting following method as described in any of the embodiments of this application.

[0008] This application provides an automatic lighting tracking method, which includes: setting up a stereo vision system in the area to be filmed; performing stereo calibration on the stereo vision system to obtain system calibration parameters; acquiring a first image set of the area to be filmed through the stereo vision system; performing stereo correction on the first image set based on the system calibration parameters to obtain a second image set; detecting a target object in the second image set and marking the position coordinates of the target object to obtain a third image set; extracting the motion features of the target based on the third image set to generate motion trajectory data of the target object; converting the motion trajectory data into lighting control parameters; and controlling the lighting to automatically follow the target object based on the lighting control parameters. In the above method, through the precise calibration and stereo correction of the stereo vision system, the precise three-dimensional coordinates of the performer can be accurately calibrated in the image. Target detection and motion trajectory extraction in the image with calibrated coordinates predict the performer's movement path, overcoming the reaction delay defects of manual operation. By converting the motion trajectory data into lighting control parameters in real time, a closed-loop control system from visual perception to lighting execution is established. Under the premise of ensuring performance safety, it realizes precise, smooth, and real-time automatic tracking of the stage lighting for the performer, significantly improving the expressiveness of stage art and production efficiency. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A schematic flowchart illustrating an automatic lighting tracking method provided in an embodiment of this application; Figure 2 This is a schematic block diagram of an automatic lighting following device provided in an embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described below with reference to the accompanying drawings.

[0012] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0013] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0014] It should be understood that in this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0015] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating an automatic lighting tracking method provided in an embodiment of this application. Figure 1 As shown, the specific steps of the automatic lighting following method include: S101-S106.

[0016] S101. Set up a stereo vision system in the area to be photographed, and perform stereo calibration on the stereo vision system to obtain the system calibration parameters.

[0017] For example, in implementing this method, a stereo vision system consisting of multiple high-resolution industrial cameras needs to be rationally arranged in the stage performance area. These cameras form a complete stereo observation network according to preset baseline distances and installation angles. Before the stereo vision system is put into operation, a professional-grade calibration board is used to acquire calibration images at multiple locations with different distances and angles within the stage space, ensuring that each stereo camera pair can completely capture the calibration board pattern. Through a precisely controlled synchronous triggering device, all cameras acquire calibration images at the same time, obtaining multiple sets of calibration image pairs. Advanced computer vision algorithms are used to automatically identify feature points on the calibration board, and the least squares method is used to iteratively optimize the camera parameters, calculating the intrinsic parameter matrix, radial and tangential distortion coefficients, and relative positional relationships between cameras for each camera. These parameters undergo rigorous reprojection error verification to ensure that the error value is less than 0.3 pixels. The resulting system calibration parameter file includes: complete camera intrinsic parameter matrices, extrinsic parameter matrices, and distortion coefficients, providing accurate geometric correction basis for subsequent image processing. The entire calibration process needs to be repeatedly verified under different lighting conditions to ensure the stability and reliability of the system in various stage environments.

[0018] S102. Obtain a first image set of the area to be photographed through a stereo vision system, and perform stereo correction on the first image set based on the system calibration parameters to obtain a second image set.

[0019] For example, all cameras in the stereo vision system are activated to synchronously acquire raw image data of the stage area at a rate of 60 frames per second. This image data is numbered and stored according to time sequence, forming a first image set containing multi-view images. The camera intrinsic parameter matrix and distortion coefficients stored in the system calibration parameter file are read, and a bilinear interpolation algorithm is used to perform distortion correction on the raw images, eliminating image distortion caused by lens optical characteristics. Based on the extrinsic parameter matrix of the stereo camera pair, the reprojection transformation matrix of the image is calculated using epipolar geometry principles, accurately projecting the left and right camera images onto the same epipolar plane to achieve perfect row alignment. The corrected images are intelligently cropped to remove black boundary areas caused by the projection transformation while maintaining the integrity of the effective content of the image. The second image set obtained after these precise processes has epipolar alignment characteristics, with corresponding points between images located on the same scan line, laying a solid foundation for subsequent stereo matching and target tracking. The entire correction process employs parallel computing technology to ensure that real-time processing performance meets the needs of stage performance.

[0020] S103. Detect the target object in the second image set and mark the position coordinates of the target object to obtain the third image set.

[0021] For example, an improved target detection algorithm is run on the second image set after stereo calibration. This algorithm integrates deep convolutional neural networks and traditional image processing techniques to accurately identify target objects such as performers in the stage scene and generate precise bounding box information. Multi-scale feature extraction technology is used to detect stable feature points within the target region. A descriptor-based stereo matching algorithm establishes accurate correspondences between the left and right images, calculating the disparity value of each feature point. Combining the camera baseline distance and focal length parameters from the system calibration parameters, the two-dimensional image coordinates are converted into three-dimensional spatial coordinates using stereo vision triangulation principles, obtaining the precise position of the target object in the stage world coordinate system. The three-dimensional spatial coordinate information of the target is deeply fused with the corresponding image frames to form a third image set containing the target's spatial location annotations. This process also includes a multi-target tracking algorithm, which ensures continuous and stable tracking of multiple performance targets in complex stage scenes through data association and state prediction, and assigns an independent identifier to each target.

[0022] S104. Extract the motion features of the target based on the third image set and generate motion trajectory data of the target object.

[0023] For example, this study delves into the target position change patterns in consecutive frame sequences within a third image set, extracting multi-dimensional motion feature parameters including displacement vector, instantaneous velocity, acceleration, and direction of motion. A dense optical flow algorithm is employed to calculate the target's motion information within the image sequence, which is then converted into a continuous motion trajectory in three-dimensional space by combining depth-sensing data. An adaptive Kalman filter-based motion state estimation model is established, capable of automatically adjusting parameters according to the target's motion characteristics to optimally estimate and smooth the target's trajectory. A temporal deep learning network is used to analyze the target's historical motion patterns, learning from the performer's movement habits and stage positioning patterns to accurately predict the target's trajectory within a one-second time window. All motion feature data is integrated according to a standardized format to generate motion trajectory data, which includes information such as timestamps, three-dimensional spatial coordinates, velocity, acceleration, direction of motion, and trajectory confidence.

[0024] S105. Convert motion trajectory data into lighting control parameters.

[0025] For example, a precise mapping relationship is constructed from the stage world coordinate system to the lighting equipment coordinate system. Homogeneous coordinate transformation converts the target's motion trajectory data into control commands for the lighting equipment. Based on the target's predicted position and motion trend, a look-ahead control algorithm calculates the required azimuth and pitch angles of the lighting pan-tilt unit, ensuring the lights can move in advance to the target's impending location. Based on the target distance and speed, fuzzy control theory is used to determine the lighting's focusing parameters, beam angle, and brightness value, achieving adaptive lighting effects. Considering the needs of stage artistic expression, a motion smoothing algorithm is introduced to optimize the control parameters, avoiding mechanical jitter or abrupt changes during lighting movement. Through parameter normalization, the control commands are converted into standardized equipment control signals, generating lighting control parameters, including azimuth control, pitch control, focus control, brightness control, and special effects control. These parameters undergo rigorous boundary checks and rationality verification to ensure both artistic expression requirements and safe equipment operation.

[0026] S106. Control the light to automatically follow the target object based on the light control parameters.

[0027] For example, standardized lighting control parameters are converted into specific device drive signals through a professional lighting control protocol and sent to the stage lighting equipment array using real-time Ethernet technology. A high-precision pan-tilt servo system is controlled to smoothly rotate according to specified azimuth and pitch angles, achieving precise control of the lighting direction. The drive current of the high-power LED light source is adjusted, and stepless adjustment of the light brightness is achieved through PWM modulation technology. An advanced optical adjustment mechanism is operated to dynamically change the illumination range, focus, and beam shape, ensuring that the light always perfectly covers the target object. Real-time acquisition of the lighting equipment's operating status data is used, and control parameters are dynamically optimized through a closed-loop control algorithm to ensure the smoothness and accuracy of light tracking. Throughout the tracking process, the movement status of the target object is continuously monitored, and the tracking strategy is intelligently adjusted according to the needs of the stage art effect, making the light movement both accurate and natural, as well as artistically expressive, achieving a perfect stage effect of harmony between the performer and the lighting.

[0028] This application provides an automatic lighting tracking method, which includes: setting up a stereo vision system in the area to be filmed; performing stereo calibration on the stereo vision system to obtain system calibration parameters; acquiring a first image set of the area to be filmed through the stereo vision system; performing stereo correction on the first image set based on the system calibration parameters to obtain a second image set; detecting a target object in the second image set and marking the position coordinates of the target object to obtain a third image set; extracting the motion features of the target based on the third image set to generate motion trajectory data of the target object; converting the motion trajectory data into lighting control parameters; and controlling the lighting to automatically follow the target object based on the lighting control parameters. In the above method, through the precise calibration and stereo correction of the stereo vision system, the precise three-dimensional coordinates of the performer can be accurately calibrated in the image. Target detection and motion trajectory extraction in the image with calibrated coordinates predict the performer's movement path, overcoming the reaction delay defects of manual operation. By converting the motion trajectory data into lighting control parameters in real time, a closed-loop control system from visual perception to lighting execution is established. Under the premise of ensuring performance safety, it realizes precise, smooth, and real-time automatic tracking of the stage lighting for the performer, significantly improving the expressiveness of stage art and production efficiency.

[0029] To more clearly illustrate the technical solution of this application, the technical solution of this application will be described below through specific embodiments. It should be noted that the specific embodiments are used to expand the description of the technical solution of this application, and are not intended to limit this application.

[0030] In some embodiments, the stereo vision system includes: a stereo camera pair; stereo calibration of the stereo vision system to obtain system calibration parameters includes: controlling two cameras in the stereo camera pair to simultaneously capture images of a preset calibration board to obtain a first calibration image and a second calibration image; the preset calibration board is placed at a predetermined distance in front of the stereo camera pair; the stereo cameras include: a left camera and a right camera; finding a first corresponding point in the first calibration image and finding a second corresponding point in the second calibration image, the first and second corresponding points forming a corresponding point pair; determining a first reference position of the second corresponding point based on the first corresponding point; calculating a camera intrinsic parameter matrix and a camera extrinsic parameter matrix based on the corresponding point pair and the first reference position; and determining system calibration parameters based on the camera intrinsic parameter matrix and the camera extrinsic parameter matrix.

[0031] For example, the stereo vision system includes: a precisely calibrated pair of stereo cameras, consisting of a left camera and a right camera, mounted at an optimal observation position above a stage while maintaining a fixed baseline distance. For stereo calibration, a specially designed checkerboard calibration board is placed three to five meters in front of the camera pair at a predetermined distance, ensuring that the calibration board completely covers the common field of view of both cameras. A synchronous trigger signal controls the left and right cameras to simultaneously capture images of the calibration board, obtaining a first calibration image and a second calibration image with strict temporal consistency. A feature point detection algorithm identifies checkerboard corner points in the first calibration image as first corresponding points, and detects corresponding corner points in the second calibration image as second corresponding points. These corner points together constitute a pair of corresponding points with a clear correspondence. Based on the geometric principles of stereo vision, the first reference position to which the second corresponding point should be adjusted is calculated based on the coordinate values ​​of the first corresponding point in the image coordinate system. This reference position reflects the theoretical coordinates under an ideal stereo configuration. Using the acquired pairs of corresponding points and their corresponding first reference positions, the camera's intrinsic and extrinsic parameter matrices are solved using a least-squares optimization algorithm. The intrinsic parameter matrix includes key parameters such as focal length, principal point coordinates, and distortion coefficients, while the extrinsic parameter matrix describes the relative positions and poses of the cameras. The calculated intrinsic and extrinsic parameter matrices are then standardized and encapsulated to form a system calibration parameter file. This file serves as the foundational parameter set for all subsequent image processing operations, ensuring that the entire vision measurement system has a unified coordinate reference system and an accurate geometric model.

[0032] In some embodiments, stereo calibration of a first image set based on system calibration parameters is performed to obtain a second image set, including: performing distortion correction on the first image set according to the camera intrinsic parameter matrix to obtain a distorted image set; determining a remapping parameter set based on the camera extrinsic parameter matrix; aligning the left and right images of the distorted image set using an epipolar correction algorithm and the remapping parameter set to obtain a corrected image set; and performing boundary cropping and size normalization on the corrected image set to generate the second image set.

[0033] For example, when performing stereo correction on the first image set based on system calibration parameters, it is necessary to read the distortion coefficients stored in the camera intrinsic parameter matrix, and use a polynomial distortion correction model to perform pixel-by-pixel coordinate transformation on each original image in the first image set to eliminate the influence of lens optical distortion on image quality, resulting in a geometrically accurate distortion-free image set. Based on the relative pose relationship between cameras described by the camera extrinsic parameter matrix, the reprojection transformation parameters of the image plane are calculated, generating a remapping parameter set containing a pixel mapping relationship matrix. Using the core principle of the epipolar correction algorithm, combined with the remapping parameter set, coordinate transformation is performed on the left and right image pairs in the distortion-reduced image set, projecting the originally non-coplanar left and right images onto the same epipolar plane, achieving strict epipolar alignment and obtaining a corrected image set with standard stereo geometric relationships. Boundary region analysis and effective content extraction are performed on the corrected image set. An intelligent cropping algorithm removes invalid boundary regions caused by the projection transformation, and the image size is standardized to ensure that all output images have uniform pixel size and aspect ratio. The second image set generated through this series of precise processing not only retains the detailed information of the original images, but more importantly, establishes accurate stereo correspondences, providing an ideal input data foundation for subsequent target detection and 3D localization.

[0034] In some embodiments, the second image set includes: a left image and a right image, each left image and each right image corresponding one-to-one. Target objects are detected in the second image set, and the position coordinates of the target objects are marked to obtain a third image set. This includes: finding pairs of corresponding points on the left and right images; determining a second reference position for the corresponding points in the right image based on the corresponding points in the left image, and recording the second reference position in a first position set; creating a first panoramic image based on the left image; creating a second panoramic image based on the right image; reading the first extrinsic parameter matrix corresponding to the right camera in the camera extrinsic parameter matrix, and performing a wrapping process on the corresponding points in the right image based on the first extrinsic parameter matrix to obtain a second position set of the second panoramic image; reading the second extrinsic parameter matrix corresponding to the left camera in the camera extrinsic parameter matrix, and performing a wrapping process on the second reference position based on the first extrinsic parameter matrix to obtain a third position set of the second panoramic image; determining a third panoramic image based on the second panoramic image, the second position set, and the third position set; and generating a third image set based on the first panoramic image and the third panoramic image.

[0035] For example, in the second image set, which includes left and right images, each left and right image maintains strict temporal synchronization and spatial correspondence. During object detection, feature point matching is first performed on the corresponding left and right images, using a local feature descriptor algorithm to find stable pairs of corresponding points. Based on the coordinates of the corresponding points in the left image's coordinate system, combined with the epipolar constraints of stereo vision, the second reference position to which the corresponding points in the right image should be adjusted is calculated. These second reference positions are recorded in the first position set according to their spatial distribution. An image stitching algorithm is used to perform panoramic fusion processing on the left image sequence, generating a first panoramic image covering the entire stage area. Similarly, a panoramic stitching operation is performed on the right image sequence to obtain a second panoramic image that corresponds spatiotemporally to the first panoramic image. The first extrinsic parameter matrix corresponding to the right camera in the camera extrinsic parameter matrix is ​​read, and the coordinates of the corresponding points in the right image are subjected to a wrap transformation based on the projection relationship described by this first extrinsic parameter matrix, resulting in a second position set in the second panoramic image coordinate system. Simultaneously, the second extrinsic parameter matrix corresponding to the left camera in the camera extrinsic parameter matrix is ​​read. Based on this second extrinsic parameter matrix, the same wrap transformation is performed on the second reference position to obtain a third set of positions in the second panoramic image coordinate system. Based on the pixel distribution characteristics of the second panoramic image, and combined with the coordinate correspondence provided by the second and third position sets, a stereo-corrected third panoramic image is generated using an image warping algorithm. The first and third panoramic images are then spatiotemporally aligned and content-fused to generate a third image set, which provides an accurate data foundation for subsequent motion feature extraction.

[0036] In the process of finding corresponding point pairs in the left and right images, a multi-stage feature matching strategy is required to ensure the accuracy and stability of the matching. First, a scale-invariant feature transform algorithm is used to detect stable keypoints in both the left and right images. These keypoints are typically located in areas with obvious texture features, such as image corners and edge intersections. A 128-dimensional feature descriptor vector is calculated for each keypoint, which comprehensively represents the texture features and gradient distribution characteristics of the image region surrounding the keypoint. The k-nearest neighbor search algorithm is used to find the best matching pair in the keypoint descriptor sets of the left and right images, and the matching similarity is evaluated by calculating the Euclidean distance between the descriptor vectors. To eliminate false matches, a bidirectional consistency check mechanism is applied, requiring that the best matching point of a keypoint in the left image is also the best matching point of that keypoint in the right image in the left image. Simultaneously, a random sampling consistency algorithm is used to verify the geometric consistency of the preliminary matching results, using fundamental matrix constraints to eliminate erroneous matching point pairs that do not conform to epipolar geometric relationships. Through this series of rigorous matching and filtering processes, a set of corresponding point pairs with high confidence is obtained. These corresponding point pairs have a clear correspondence in the left and right images, providing reliable input data for subsequent stereo vision calculations. Each successful corresponding point pair not only contains the pixel coordinate information of the corresponding points in the left and right images, but also records additional attributes such as the scale, orientation, and matching confidence of the feature points. These attributes will play an important role in subsequent processing steps.

[0037] As a core technology in image geometric processing, rollover transformation requires the construction of a projection transformation model based on the camera's extrinsic parameters. This model uses homogeneous coordinate representation and describes the rigid body transformation relationships in three-dimensional space through a 4x4 transformation matrix. When processing corresponding points in the right image, the first extrinsic parameter matrix corresponding to the right camera is first read. This matrix contains important parameters such as rotation matrix and translation vector, accurately describing the pose relationship of the right camera relative to the world coordinate system. The image coordinates of the corresponding points are mapped to three-dimensional space through inverse perspective transformation, and then projected into the panoramic coordinate system according to the transformation relationship defined by the first extrinsic parameter matrix. This process involves complex matrix operations and coordinate transformations.

[0038] In the specific calculation process, each pixel needs to undergo a rigorous coordinate transformation process. First, the two-dimensional image coordinates are expanded into homogeneous coordinates, and then multiplied with the projection transformation matrix to obtain temporary coordinates in the target coordinate system. Next, perspective division is performed to convert the homogeneous coordinates back to standard two-dimensional coordinates. Due to the discrete nature of digital images, the calculated coordinate values ​​are often not integers. In this case, a bilinear interpolation algorithm must be used to calculate the color value of the target pixel. This interpolation algorithm requires taking the color values ​​of four adjacent pixels in the source image and performing a weighted average. The weights are determined based on the relative distance between pixels to ensure a smooth and natural transformation effect.

[0039] When processing the roll transformation at the second reference position, the same transformation parameters but a different calculation path are required. The second extrinsic parameter matrix corresponding to the left camera is read, and the second reference position is treated as an ideal projection point for inverse transformation calculation. This process requires special attention to coordinate system consistency, ensuring all transformations are performed within a unified world coordinate system. Through precise roll transformation processing, the final sets of second and third positions establish an accurate correspondence in the panoramic coordinate system, laying a solid foundation for subsequent image correction and stereo matching. The entire transformation process also needs to consider technical details such as image boundary processing and blank area filling to ensure that the final generated panoramic image has complete visual information and accurate geometric relationships.

[0040] In some embodiments, extracting motion features of the target based on a third image set and generating motion trajectory data of the target object includes: uniformly dividing the third panoramic image into grids to obtain grid division results, the grid division results including: a first position of a grid corner point, the position of a related corresponding point within a predetermined range around the grid corner point, and a reference position of the related corresponding point; determining a second position of the grid corner point based on the first position, the position of the related corresponding point, and the reference position of the related corresponding point; determining the affine transformation relationship of each grid based on the first position and the second position; aggregating all affine transformation relationships to generate a stereo correction template; and calculating the motion features of the target object based on the stereo correction template and the third image set to generate motion trajectory data.

[0041] For example, when extracting target motion features based on a third image set, the third panoramic image needs to be uniformly divided into grids, resulting in a grid division result containing grid corner coordinate information. This grid division result records the first position coordinates of each grid corner point and also statistically analyzes the spatial position information and corresponding reference position coordinates of all associated points within a predetermined range around each grid corner point. Based on the first position coordinates of the grid corner point, the current position coordinates of the associated points, and the reference position coordinates of the associated points, a weighted average algorithm is used to calculate the second position coordinates that each grid corner point should be adjusted to. Based on the correspondence between the first and second position coordinates of the grid corner points, an affine transformation model is used to calculate the geometric transformation parameters of each grid unit, determining the affine transformation relationship of each grid. The affine transformation relationship parameters of all grid units are collected to construct a stereo correction template covering the entire image region, which fully describes the spatial transformation mapping relationship of image pixels. By combining the geometric transformation information provided by the stereo correction template and the temporal image data of the third image set, the motion characteristics of the target object, such as motion vector, motion speed and motion direction, are calculated between consecutive frames using a motion analysis algorithm to generate motion trajectory data. The motion trajectory data includes: timestamp, three-dimensional coordinates, motion state parameters and confidence index.

[0042] When uniformly dividing a third panoramic image into grids, it is necessary to determine appropriate grid size parameters based on image resolution and accuracy requirements. Typically, the image is divided into several uniformly sized rectangular grid cells. Each grid cell contains four grid corner points, arranged in row and column order to form a regular grid structure. During grid division, it is crucial to ensure that the grid boundaries align with the image boundaries, while avoiding excessive computation due to overly small grid sizes or impacting correction accuracy due to overly large grid sizes. When collecting the spatial location information of all associated points within a predetermined range around each grid corner point, a circular or rectangular search area needs to be established centered on the grid corner point, and all associated points falling within this area are counted. Each associated point contains two sets of data: current position coordinates and reference position coordinates. The current position coordinates represent the actual position of the point in the original image, while the reference position coordinates represent the ideal position the point should reach after stereo correction. By calculating the difference between the current position coordinates and the reference position coordinates, the position correction vector for each associated point can be obtained. When calculating the second position coordinates of grid corner points using a weighted average algorithm, an appropriate weight coefficient needs to be assigned to the position correction vector of each associated corresponding point. This weight coefficient is typically inversely proportional to the distance from the associated corresponding point to the grid corner point. This distance-weighted calculation method ensures that associated corresponding points closer to the grid corner point have a greater impact on the calculation results, while those farther away have a smaller impact, thus guaranteeing the local smoothness and global consistency of the grid transformation. The final set of second position coordinates of the grid corner points fully describes the spatial transformation that should be performed on the image pixels, laying a solid foundation for subsequent affine transformation calculations.

[0043] In some embodiments, the second position of the grid corner point is determined based on the first position, the position of the associated corresponding point, and the reference position of the associated corresponding point. The specific calculation formula is as follows: ; in, For the second position, As the first position, To correlate the position of a point with the same name with the position difference of the second position of a grid corner point, The decay function, The distance between the first position and the position of the associated corresponding point is denoted by n, where n is the number of associated corresponding points.

[0044] In the specific implementation process, the attenuation function Typically, a Gaussian kernel function or an exponential decay function is used. This ensures that related points closer to the grid corner have a greater influence on the calculation results, while the influence of more distant related points gradually decreases with distance. This weighted calculation method guarantees that the new coordinates of the grid corner fully consider the positional correction information of all surrounding related points, while maintaining the overall smoothness and continuity of the grid transformation. This effectively avoids local distortions and abrupt changes that may occur during image transformation, providing a reliable geometric transformation basis for subsequent stereo correction and motion analysis.

[0045] In some embodiments, converting motion trajectory data into lighting control parameters includes: establishing a spatial mapping relationship based on the target motion trajectory data, and generating a lighting control instruction set according to the spatial mapping relationship.

[0046] For example, in the process of converting motion trajectory data into lighting control parameters, it is first necessary to establish an accurate spatial mapping relationship based on the target motion trajectory data. This mapping relationship defines a mathematical transformation model from the target's three-dimensional motion space to the lighting equipment control space. By analyzing information such as the target position sequence, motion velocity vector, and motion direction angle in the motion trajectory data, and combining the physical characteristic parameters and mechanical motion constraints of the stage lighting equipment, a nonlinear mapping function describing the correspondence between the target's motion state and the lighting control parameters is constructed. Based on the established spatial mapping relationship, the real-time acquired target motion trajectory data is converted into specific equipment control commands, generating a lighting control command set. The lighting control command set includes multi-dimensional control parameters such as pan / tilt azimuth angle, pan / tilt pitch angle, focus parameters, brightness value, and beam angle. This lighting control command set is encapsulated in a standardized format and can be directly sent to the stage lighting equipment via standard lighting control protocols such as DMX512 or Art-Net to drive the lighting equipment to perform corresponding actions, achieving a precise, smooth, and real-time automatic tracking effect of the light on the moving target. This ensures that the light movement maintains high positioning accuracy while possessing good motion smoothness and artistic expression throughout the tracking process.

[0047] Please see Figure 2 , Figure 2 This is a schematic block diagram of an automatic lighting follower device 200 provided in an embodiment of this application. The automatic lighting follower device 200 is used to execute the aforementioned automatic lighting follower method. The automatic lighting follower device 200 can be configured in a server.

[0048] The server can be a standalone server, a server cluster, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0049] like Figure 2 As shown, the automatic lighting following device 200 includes: a parameter calibration module 201, an image acquisition module 202, a position calibration module 203, a trajectory synthesis module 204, a parameter conversion module 205, and an instruction generation module 206.

[0050] The parameter calibration module 201 is used to set up a stereo vision system in the area to be photographed and to perform stereo calibration on the stereo vision system to obtain the system calibration parameters.

[0051] The image acquisition module 202 is used to acquire a first image set of the area to be photographed through a stereo vision system, and to perform stereo correction on the first image set based on the system calibration parameters to obtain a second image set.

[0052] The position calibration module 203 is used to detect target objects in the second image set and mark the position coordinates of the target objects to obtain the third image set.

[0053] The trajectory synthesis module 204 is used to extract the motion features of the target based on the third image set and generate motion trajectory data of the target object.

[0054] The parameter conversion module 205 is used to convert motion trajectory data into lighting control parameters.

[0055] The instruction generation module 206 is used to control the illumination direction and illumination characteristics of the lighting device in real time through the motor drive system and the light source adjustment system based on the lighting control instruction set, so as to realize the automatic tracking of the light to the target.

[0056] This application provides a lighting device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, implement the automatic lighting following method as described in any of the embodiments of this application.

[0057] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it causes the processor to implement an automatic lighting following method as described in any of the embodiments of this application.

[0058] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for automatic following of lighting light, characterized in that, The method includes: A stereo vision system is set up in the area to be photographed, and the stereo vision system is stereo calibrated to obtain the system calibration parameters; The first image set of the area to be photographed is obtained through the stereo vision system, and the first image set is stereo corrected based on the system calibration parameters to obtain the second image set; The target object is detected in the second image set, and the position coordinates of the target object are marked to obtain the third image set; Based on the third image set, the motion features of the target are extracted to generate the motion trajectory data of the target object; The motion trajectory data is converted into lighting control parameters; Based on the aforementioned lighting control parameters, the lighting is controlled to automatically follow the target object.

2. The automatic lighting tracking method as described in claim 1, characterized in that, The stereo vision system includes: a stereo camera pair; the stereo calibration of the stereo vision system to obtain system calibration parameters includes: The two cameras in the stereo camera pair are controlled to simultaneously capture images of a preset calibration board to obtain a first calibration image and a second calibration image. The preset calibration board is placed at a predetermined distance in front of the stereo camera pair. The stereo camera includes a left camera and a right camera. Find a first corresponding point on the first calibration image and find a second corresponding point on the second calibration image. The first corresponding point and the second corresponding point constitute a pair of corresponding points. Determine the first reference position of the second corresponding point based on the first corresponding point; Calculate the camera intrinsic parameter matrix and camera extrinsic parameter matrix based on the corresponding point pairs and the first reference position; The system calibration parameters are determined based on the camera intrinsic parameter matrix and the camera extrinsic parameter matrix.

3. The automatic lighting tracking method as described in claim 2, characterized in that, The step of performing stereo correction on the first image set based on the system calibration parameters to obtain the second image set includes: The first image set is subjected to distortion correction processing based on the camera intrinsic parameter matrix to obtain a distortion-corrected image set; The remapping parameter set is determined based on the camera extrinsic matrix; The left and right images of the distortion-reduced image set are aligned using the epipolar correction algorithm and the remapping parameter set to obtain the corrected image set; The corrected image set is then cropped at the boundaries and its size is normalized to generate a second image set.

4. The automatic lighting tracking method as described in claim 2, characterized in that, The second image set includes: a left image and a right image, with each left image and each right image corresponding one-to-one. The step of detecting a target object in the second image set and marking the position coordinates of the target object yields a third image set, including: Find pairs of points with the same name on the left and right images; Determine the second reference position of the corresponding point in the right image based on the corresponding point in the left image, and record the second reference position in the first position set; A first panoramic image is generated based on the left image; A second panoramic image is generated based on the right image; Read the first extrinsic matrix corresponding to the right camera in the camera extrinsic matrix, and perform wrapping processing on the corresponding points of the right image according to the first extrinsic matrix to obtain the second position set of the second panoramic image; Read the second extrinsic matrix corresponding to the left camera in the camera extrinsic matrix, and perform a wrapping process on the second reference position according to the first extrinsic matrix to obtain the third position set of the second panoramic image; A third panoramic image is determined based on the second panoramic image, the second location set, and the third location set; A third image set is generated based on the first panoramic image and the third panoramic image.

5. The automatic lighting tracking method as described in claim 4, characterized in that, The step of extracting motion features of the target based on the third image set and generating motion trajectory data of the target object includes: The third panoramic image is uniformly divided into grids to obtain grid division results. The grid division results include: the first position of the grid corner point, the position of the associated corresponding point within a predetermined range around the grid corner point, and the reference position of the associated corresponding point. The second position of the grid corner point is determined based on the first position, the position of the associated corresponding point, and the reference position of the associated corresponding point; The affine transformation relationship for each grid is determined based on the first position and the second position; By combining all the aforementioned affine transformation relationships, a stereo correction template is generated; Based on the stereo correction template and the third image set, the motion features of the target object are calculated, and motion trajectory data is generated.

6. The automatic lighting tracking method as described in claim 5, characterized in that, The second position of the grid corner point is determined based on the first position, the position of the associated corresponding point, and the reference position of the associated corresponding point. The specific calculation formula is as follows: ； in, For the second position, For the first position, The difference between the position of the associated corresponding point and the second position of the grid corner point. The decay function, The distance between the first position and the position of the associated corresponding point is denoted as n, where n is the number of the associated corresponding points.

7. The automatic lighting tracking method as described in claim 1, characterized in that, The step of converting the motion trajectory data into lighting control parameters includes: A spatial mapping relationship is established based on the target motion trajectory data, and a lighting control command set is generated according to the spatial mapping relationship.

8. An automatic lighting tracking device, characterized in that, The automatic lighting follower device is used to execute the automatic lighting follower method as described in any one of claims 1-7, and the automatic lighting follower device includes: The parameter calibration module is used to set up a stereo vision system in the area to be photographed and to perform stereo calibration on the stereo vision system to obtain system calibration parameters. The image acquisition module is used to acquire a first image set of the area to be photographed through the stereo vision system, and to perform stereo correction on the first image set based on the system calibration parameters to obtain a second image set; A location calibration module is used to detect target objects in the second image set and mark the location coordinates of the target objects to obtain a third image set; The trajectory synthesis module is used to extract the motion features of the target based on the third image set and generate the motion trajectory data of the target object; The parameter conversion module is used to convert the motion trajectory data into lighting control parameters; The instruction generation module is used to control the illumination direction and illumination characteristics of the lighting device in real time through the motor drive system and the light source adjustment system based on the lighting control instruction set, so as to realize the automatic tracking of the light to the target.

9. A lighting device, characterized in that, The lighting device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the automatic lighting following method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the automatic lighting following method as described in any one of claims 1 to 7.

Images