Acquisition offset detection method, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention provides an acquisition offset detection method, electronic equipment and a computer readable storage medium. The method comprises the following steps: carrying out image acquisition on a target scene provided with a plurality of markers to obtain a target image; determining a plurality of candidate frames respectively comprising a plurality of markers in the target image, and confidence coefficients respectively corresponding to the plurality of candidate frames; the confidence coefficient is determined based on the relative position among the plurality of markers in the corresponding candidate frame; determining a target frame with the maximum confidence in the plurality of candidate frames; and based on the current coordinates corresponding to the plurality of markers in the target frame and the preset initial coordinates of the plurality of markers, determining an acquisition offset result of image acquisition of the target scene. According to the technical scheme provided by the embodiment of the invention, the technical problems that the camera offset detection efficiency is not ideal and misjudgment is easy to occur in the related technology are solved.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to a method for detecting acquisition offset, an electronic device, and a computer-readable storage medium. Background Technology

[0002] In video surveillance systems, for specific monitoring scenarios such as airport security checkpoints and park entrances / exits, it is usually necessary to fix the position and angle of the camera lens to monitor specific areas in real time. However, if the monitoring lens is shifted from its original position or angle due to malicious damage or environmental factors, the actual image captured by the lens will deviate from the area to be monitored. In this case, the security monitoring system is prone to missed detections or false alarms, posing a security risk.

[0003] Existing technologies typically employ manual visual inspection or feature comparison to detect camera lens shift. Manual visual inspection involves system administrators reviewing real-time camera feeds and comparing the current image with a preset image to determine if the camera has shifted. Feature comparison first extracts edge features, local features, or neural network features from the current image and historical images, then compares these features to determine if the camera has shifted. However, manual visual inspection suffers from low efficiency and high labor costs due to the limitations of human vision, and is prone to missed detections. Feature comparison, due to the complexity and variability of monitoring scenarios, suffers from low generalization ability and cannot adapt to various monitoring situations, especially under extreme conditions where it is prone to missed or false detections. Therefore, existing technologies suffer from unsatisfactory efficiency and accuracy in shift detection. Summary of the Invention

[0004] This application provides a method for acquiring offset detection, an electronic device, and a computer-readable storage medium to alleviate or solve the technical problems in related technologies, such as unsatisfactory camera offset detection efficiency and easy misjudgment.

[0005] In a first aspect, embodiments of this application provide a method for acquiring offset detection, including: Image acquisition is performed on a target scene with multiple markers to obtain a target image; In the target image, multiple candidate bounding boxes, each containing multiple markers, are identified, along with the confidence scores corresponding to each candidate bounding box. The confidence scores are determined based on the relative positions of the multiple markers within the corresponding candidate bounding box. Determine the target bounding box with the highest confidence among multiple candidate bounding boxes; Based on the current coordinates of multiple markers in the target bounding box and the preset initial coordinates of the multiple markers, the acquisition offset result for image acquisition of the target scene is determined.

[0006] Secondly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the methods of embodiments of this application when executing the computer program.

[0007] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method of any one of the embodiments of this application.

[0008] Based on the acquisition offset detection method described in the first aspect above, this application has at least the following beneficial effects or advantages: By setting multiple markers in the target scene, these markers can serve as key reference points after image acquisition. When determining candidate bounding boxes, the confidence level is calculated based on the relative positions between the markers, making the marker positioning more accurate. Compared to methods that rely solely on marker positions for offset detection, this avoids misjudgments caused by complex image backgrounds, image noise, and other factors. The candidate bounding box with the highest confidence level is selected as the target bounding box, ensuring that the coordinates of the markers it contains are the most reliable. Through the automatic calculation of markers and confidence levels, the number of manual steps is greatly reduced, improving the system's automation level and enhancing the processing efficiency and detection accuracy for lens offset detection.

[0009] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0010] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0011] Figure 1 A flowchart of the acquisition offset detection method according to an embodiment of this application is shown; Figure 2 A schematic diagram of the markers in the acquisition offset detection method of this application is shown; Figure 3 A flowchart illustrating the acquisition offset detection method according to an embodiment of this application is shown; Figure 4 A block diagram of the acquisition offset detection device according to an embodiment of this application is shown; Figure 5 A block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0012] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0013] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0014] The following terms will be used in the following text: RetinaFace is a deep learning-based face detection model that can simultaneously perform face detection, key point localization (such as the corners of the eyes and the tip of the nose), and pixel-level mask prediction. It is often used in multi-scale face detection.

[0015] ResNet18 is a deep residual network. Its core is the residual block. By introducing skip connections, it solves the gradient vanishing problem in deep network training, enabling the network to train deeper layers, including 18 layers. It has good feature extraction capabilities and high computational efficiency.

[0016] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0017] Figure 1 A flowchart of the acquisition offset detection method according to an embodiment of this application is shown, as follows: Figure 1 As shown, the method may include steps S101 to S104.

[0018] Step S101: Acquire images of the target scene with multiple markers to obtain the target image; Step S102: Determine multiple candidate bounding boxes, each containing multiple markers, in the target image, and the confidence scores corresponding to each candidate bounding box; the confidence scores are determined based on the relative positions of the multiple markers within the corresponding candidate bounding boxes. Step S103: Determine the target bounding box with the highest confidence among multiple candidate bounding boxes; Step S104: Based on the current coordinates of multiple markers in the target bounding box and the preset initial coordinates of the multiple markers, determine the acquisition offset result for image acquisition of the target scene.

[0019] For example, cameras, video cameras themselves, and shooting equipment mounted on drones can directly perform image acquisition and monitoring operations. Among the aforementioned hardware devices, cameras can take various forms, including depth cameras and industrial cameras, while video cameras can be PTZ cameras, surveillance cameras, network cameras (IPCs), etc.

[0020] The aforementioned executing entity can be an image processing system installed in the hardware device, or it can be an integrated system combining hardware and software. Image processing systems, such as camera firmware, video surveillance platform software, and industrial vision inspection software, are responsible for controlling hardware acquisition actions, processing image data, and achieving real-time monitoring functions. They can also be intelligent terminals or cloud computing platforms with edge computing capabilities, which can remotely control hardware and execute data processing tasks via a network. Integrated systems combining hardware and software, such as security monitoring systems, industrial automation inspection systems, and visual perception systems for autonomous vehicles, achieve image acquisition and monitoring targets through the coordinated use of hardware and software.

[0021] For example, the target scenarios mentioned above could be security checkpoints, park / residential area entrances / exits, ATMs, counter monitoring areas, warehouse and logistics goods sorting areas, factory assembly line quality inspection points, urban road manhole covers, bridge structure monitoring points, etc. For security checkpoints and park / residential area entrances / exits, markers can be placed around the security equipment or in fixed positions on the wall to ensure the camera is always focused on the core area of ​​the security checkpoint. For ATMs, markers can be fixed to the ATM body or counter in an unobstructed position. For factory assembly line quality inspection points, cameras need to monitor the product production process in real time, and markers are placed at fixed workstations on the assembly line, such as parts assembly points or defect detection areas. For logistics goods sorting areas, markers are used to monitor the goods sorting process, and can be placed on the edge of sorting equipment or conveyor belts. For urban road manhole cover and bridge structure monitoring point scenarios, the markers can be fixed to the edge of the manhole cover or at key bridge structures. The camera determines whether the lens is shifted by detecting changes in the marker's position, and can also assist in monitoring facility anomalies, such as manhole cover displacement or bridge cracks.

[0022] When the acquisition device is a camera, the target image is a video frame in the acquired video stream, which can be an image of the current frame extracted from the camera video stream at fixed intervals.

[0023] In the embodiments provided in this application, image acquisition is performed on a target scene with multiple markers to obtain a target image. A target detection algorithm is used to identify multiple candidate bounding boxes that may contain markers in the image. The confidence score of each bounding box is dynamically generated by calculating the matching degree between the actual relative positions of the markers within each candidate bounding box and a preset geometric relationship. This confidence score effectively reflects the reliability of the detection results. The target bounding box with the highest confidence score is selected to ensure that subsequent calculations are based on the most reliable detection data. By comparing the current coordinates of the markers within the target bounding box with preset initial coordinates, the composite offset of the camera, such as translation, rotation, or scaling, is accurately calculated.

[0024] Image acquisition is performed on a target scene with multiple markers to obtain a target image. Multiple candidate bounding boxes containing each marker and their corresponding confidence scores are determined in the target image. The confidence scores are determined based on the relative positions of the markers within the candidate bounding boxes. The target bounding box with the highest confidence score is selected from the multiple candidate bounding boxes. The offset result of image acquisition is determined based on the current coordinates of the markers in the target bounding box and the preset initial coordinates.

[0025] For example, the marker is a reference point used for camera lens offset detection. Figure 2 This illustration shows a schematic diagram of the markers used in the acquisition offset detection method according to an embodiment of this application. For example, the marker sample... Figure 2 As shown, Five key points for the marker, , , , The proportions of the four sides The actual dimensions of the aforementioned signage can be adjusted according to the specific scenario.

[0026] After placing the marker, record five key points of the marker. The initial coordinate information in the image captured by the lens is represented as follows: , , , , ,in and Representing points respectively In the image shaft and The coordinates of the axes are similar, and so on for the rest.

[0027] Manual visual inspection is inefficient and costly due to the limitations of human vision, and is prone to missed detections. Feature comparison, on the other hand, suffers from low generalization ability due to the complexity and variability of monitoring scenarios, and cannot adapt to various monitoring scenarios. In particular, it is prone to missed detections and false detections under extreme conditions, such as when the background feature recognition is low, and when it is prone to false detections in severe weather or under sudden changes in lighting.

[0028] To address the aforementioned problems, according to the embodiments provided in this application, before performing step S102: determining multiple candidate bounding boxes that each includes multiple markers in the target image, and the confidence levels corresponding to the multiple candidate bounding boxes, the method further includes the following steps: An image quality recognition model is used to determine the image quality of the target image. The image quality recognition model is trained on historical images under different weather conditions and different lighting conditions. If the image quality is greater than a predetermined threshold, the process of determining multiple candidate bounding boxes in the target image is allowed.

[0029] In the embodiments provided in this application, the image quality of the target image is first determined using an image quality recognition model in the camera lens offset detection process. This image quality recognition model is trained on historical images under different weather and lighting conditions. Through learning from a large amount of image data under complex environments, it can accurately judge the quality of images. Once the target image is acquired, it is input into the trained image quality recognition model. The model then comprehensively analyzes multiple dimensions of the image, such as sharpness, brightness, contrast, and noise, and outputs the corresponding image quality.

[0030] The image quality is compared to a predetermined threshold. If the image quality exceeds the threshold, it means the current target image quality is good enough to meet the accuracy requirements of subsequent processing. Only then is it allowed to perform the process of determining multiple candidate bounding boxes in the target image. If a poor-quality image is used for subsequent analysis, such as in bad weather causing blurriness, insufficient lighting causing darkness, or strong light causing overexposure, the features of landmarks in the image may be difficult to extract and identify accurately, easily leading to false positives or false negatives. Only when the image quality meets the standard can the subsequent operation of determining candidate bounding boxes by analyzing the position of landmarks in the target image, and then determining whether the camera lens is offset, be reliable and accurate.

[0031] The image quality recognition model, trained on historical images from complex environments, effectively adapts to image acquisition conditions in various real-world scenarios, avoiding interference from weather and lighting conditions, thus significantly improving the system's robustness. By setting image quality thresholds to control subsequent processing, the quality of input data is guaranteed from the source, reducing erroneous judgments caused by poor image quality, lowering the probability of false positives and false negatives, and making camera lens shift detection results more accurate and reliable. Ultimately, this effectively ensures the stability and security of applications such as video surveillance systems, reduces the false negative and false negative rates of camera lens shift detection under extreme conditions, and effectively improves the efficiency and accuracy of camera lens shift detection.

[0032] For example, the different weather conditions mentioned above, such as sunny days, rainy days, foggy / hazy days, and snowy days, and the different lighting conditions, can include strong light, weak light, backlight, and nighttime lighting. In sunny and strong light conditions, reflections on the sign's surface cause local pixel saturation, and the edge contours may be distorted by shadows, potentially leading to coordinate positioning deviations of the sign and increased edge detection errors. In rainy days, raindrops obscure the sign, blurring the image and attenuating color information, potentially causing motion blur (raindrops falling), blurred sign edges, and coordinate frame positioning deviations, with the possibility of misidentifying raindrops as noise. In snowy days, large areas of white snow reduce the contrast between the sign and the background, and falling snowflakes create dynamic image noise, making the image susceptible to background interference and causing coordinate deviations of the sign. In foggy / hazy days, image fogging blurs the edges of the sign, loses texture details, and reduces color saturation, making feature extraction difficult in image processing, reducing coordinate positioning accuracy, and leading to missed detections of relatively small signs. In backlit conditions, because the main body of the marker is in shadow and its outline is unclear, only the edge outline may be preserved while internal features are lost, making it impossible to accurately extract the complete shape of the marker, and the coordinates may only include a partial area. In low light and low-light conditions at night, image noise increases, the pixel signal-to-noise ratio is low, the details of the marker are easily buried by noise, the coordinate frame position is offset, and even false coordinates are generated due to noise.

[0033] For example, the image quality recognition models mentioned above can be of various types, such as ResNet18, DenseNet (densely connected network), MobileNet series (lightweight network), and ViT (Vision Transformer). ResNet18 solves the gradient vanishing problem caused by increased network depth in traditional CNNs through residual connections, effectively extracting low-level textures (such as edges and noise) and high-level semantic features (such as scene structure and object outlines) from images. In image quality recognition, these features can be used to determine whether an image is blurred, overexposed, or subject to noise interference due to weather, lighting, or other factors. For example, images in severe weather may have raindrop blur or fog obscuring the image; ResNet18 can identify such quality defects by extracting abnormal patterns in pixel distribution. DenseNet enhances information flow by transferring features between layers through dense connections, making it suitable for capturing subtle quality differences in images and exhibiting good adaptability to mild blurring or noise. MobileNet uses depthwise separable convolution to reduce computation and has a small model size. ViT is used to apply the Transformer architecture to image tasks. It captures global pixel dependencies through a self-attention mechanism, making it suitable for analyzing long-range structural distortions in images and well-suited for tasks such as global fog coverage.

[0034] For example, the preprocessed image is input into a trained ResNet18 image quality recognition network model. This model extracts image features, such as texture, edges, and contrast, through a multi-layer residual network, and outputs a quality score (QS) representing the current image. The QS ranges from [0,1], with a higher value indicating higher image quality and vice versa. If the QS of the current image is greater than a set threshold, the next step is performed for subsequent camera lens shift detection analysis; otherwise, the image quality recognition is repeated. For example, a clear image on a sunny day might score 0.9, while a blurry image on a rainy day might only score 0.3. A preset quality threshold (e.g., 0.7) is used. When the QS exceeds the threshold, it indicates that the image quality meets the standard, and the subsequent lens shift detection process can proceed to ensure that the analysis is based on clear and reliable image data. If the QS does not reach the threshold, it indicates that the image quality is poor due to factors such as weather and lighting. In this case, the system automatically repeats the image quality recognition until a qualified image is obtained or an abnormality prompt is triggered.

[0035] For example, after acquiring the target image through the acquisition device, the image is preprocessed before subsequent processing. Preprocessing methods can be various, such as image scaling, normalization, and standardization. Image scaling adjusts the original image to a fixed size (e.g., 224×224, 300×300) to meet the input requirements of subsequent models. By standardizing the input dimension, deep learning models (such as ResNet and ViT) typically require fixed-size inputs to avoid fluctuations in computational complexity due to varying image sizes. This approach balances computational efficiency with feature preservation; excessively small sizes may lose details (such as landmark edges), while excessively large sizes increase computational consumption. Therefore, an appropriate scaling ratio must be selected based on model requirements, retaining the minimum size necessary to preserve key landmark features.

[0036] Normalization standardizes image pixel values, mapping them to the [0,1] interval (e.g., by dividing by 255) for linear normalization.

[0037] Standardization by subtracting the mean and dividing by the standard deviation ensures that pixel values ​​conform to a normal distribution (mean 0, variance 1). This approach effectively addresses the issue of higher pixel values ​​under strong light, preventing gradient updates from becoming unstable due to variations in pixel value ranges and improving training and inference efficiency. It also eliminates the impact of lighting differences across different scenes on pixel values, allowing the model to focus more on structural features (such as landmark shapes) rather than brightness fluctuations.

[0038] According to the embodiments provided in this application, in step S102: determining the confidence levels corresponding to the multiple candidate bounding boxes may include the following processing: The proportion of each marker is determined based on the area of ​​the bounding rectangle of multiple markers and the image area of ​​the target image. The confidence levels of multiple candidate bounding boxes are determined based on the proportion of each marker and the relative positions of the markers.

[0039] In the embodiments provided in this application, when determining the confidence scores of multiple candidate bounding boxes, the ratio of the area of ​​the bounding rectangle of multiple markers to the area of ​​the target image is first calculated to obtain the marker proportion. Based on the marker proportion and the relative positions between markers, the confidence score of each candidate bounding box is determined. The marker proportion measures its salience in the image; a reasonable proportion leads to more reliable detection, avoiding difficulties in feature extraction due to excessively small proportions or excessive proportions exceeding the field of view. Combining the relative positions between markers (such as preset side length ratios and geometric relationships) verifies the rationality of the marker layout within the candidate bounding box and filters out erroneous bounding boxes caused by occlusion, noise, etc. The combination of these two methods makes the confidence score calculation more comprehensive, improves the accuracy of candidate bounding box selection, and ensures that subsequent offset detection is based on real and valid marker positions, reducing misjudgments caused by environmental interference and improving the accuracy and stability of lens offset detection.

[0040] For example, a camera is used as the acquisition device. After the camera is installed and debugged, a suitable location needs to be selected within the camera lens's field of view to place a marker. The marker placement must meet two conditions: the marker must be clearly visible in the captured image and will not be obstructed during the monitoring period; the area of ​​the marker in the captured image must be proportional to the total area of ​​the image. ,in and Representing the edges of the marker and edge Length in the target image and These represent the length and width of the target image, respectively. The preferred ratio value can be set to other values ​​according to specific needs.

[0041] According to the embodiments provided in this application, a marker detection and localization model is used to output multiple candidate bounding boxes. This model can be a variety of network models with keypoint detection capabilities, such as the RetinaFace model, the MTCNN (Multi-task Cascaded Convolutional Networks) model, the YOLOv8 model, etc. The RetinaFace model can simultaneously achieve efficient marker detection and high-precision keypoint localization, and is highly adaptable to complex scenes. The MTCNN model generates and optimizes candidate bounding boxes and keypoints in stages through cascaded convolutional networks; its lightweight design is suitable for rapid detection of small-scale markers. The YOLOv8 model integrates object detection and keypoint detection heads, outputting candidate bounding boxes end-to-end, and possesses both ultra-high-speed inference and custom marker detection capabilities. Preferably, the RetinaFace model is used for processing.

[0042] Under extreme conditions, such as when the background features of the scene have low recognizability, manual visual inspection is prone to overlooking subtle shifts due to visual fatigue. Feature comparison, on the other hand, suffers from the lack of distinguishable edges or local features in the background, leading to the model's inability to accurately extract differences and resulting in missed detections. A marker detection and localization approach is adopted. By setting markers with clear geometric features in the scene (such as a five-point structure OABCD with a preset side length ratio), the detection task is transformed into locating the key points of the markers and their relative relationships. This is similar to the recognition logic for fixed features such as eyes and nose tip in facial keypoint detection. Utilizing the structured features designed by the markers themselves (such as a side length ratio of OA∶OB∶OC∶OD=1∶2∶4∶3), and through the feature extraction capabilities of neural networks based on models such as RetinaFace, the bounding rectangle and keypoint coordinates are accurately identified.

[0043] By transforming passive detection that relies on background features into active localization that relies on landmarks—similar to how human pose estimation determines movement through keypoint positions—landmark detection establishes strong prior constraints through pre-defined geometric relationships (such as a fixed shape formed by five points). When the camera shifts, the coordinates of the landmark in the image change according to geometric rules. For example, even in foggy conditions with a blurred background, RetinaFace can still locate keypoints through the outline and relative positional relationships of landmarks, thus providing a stable basis for camera shift detection and ensuring detection reliability in complex environments.

[0044] For example, the target image is input into the trained RetinaFace landmark detection and localization model, and the output is a set of possible landmark bounding boxes (i.e., multiple candidate bounding boxes) in the current image. ,in This indicates the total number of target borders to be output. Indicates the border number.

[0045]

[0046] in,( )and( ) respectively represent the first The coordinates of the top left and bottom right corners of the border in the image. Indicates the first The confidence level of each border, ( ), ( ), ( ), ( ), They represent the first Key points corresponding to each border Coordinates in the image, then from the bounding box set Select confidence level The largest border is taken as the border of the detected landmark in the current image, denoted as: .

[0047] According to the embodiments provided in this application, based on the current coordinates of multiple markers in the target bounding box and the preset initial coordinates of the multiple markers, the acquisition offset result for image acquisition of the target scene is determined, including: Determine the coordinate deviation between the current coordinates and the initial coordinates of multiple markers; The pixel offset is determined based on the predetermined marker size; The acquisition offset result is determined based on the coordinate deviation and pixel offset.

[0048] In the embodiments provided in this application, the coordinate deviation between the current coordinates and preset initial coordinates of multiple markers in the target bounding box is calculated to quantify the positional change of each marker. Pixel offsets are determined based on the marker dimensions, and these pixel offsets are further converted into physically meaningful actual offsets. The physical offsets of all markers are then used to calculate the final acquisition offset result through weighted averaging or affine transformation. Physical size calibration eliminates the influence of lens focal length and shooting distance, making the detection results from different camera positions comparable.

[0049] For example, the deviation between the current position coordinates and the initial position coordinates of the key point of the marker is further calculated according to the following formula. (i.e., the acquired offset result), if If the distance exceeds a set threshold, the camera lens is considered to have shifted; otherwise, no shift has occurred. The distance between two points from multiple markers can be selected as a reference. For example, using a predetermined multiple of the initial OA side length as a reference, the total pixel offset is converted into a relative proportion. This predetermined multiple is the magnification ratio, ensuring the offset falls within a more easily interpretable range.

[0050]

[0051] in, The sign indicates the symbol, indicating Each point in, and Indicates the initial coordinates. and Indicates the current coordinates. and Represent the x-coordinate and y-coordinate of point A. and Represents the x-coordinate and y-coordinate of point O.

[0052] According to the embodiments provided in this application, markers can be of many types, broadly categorized into two types: physical and non-physical. Physical markers are fixed, physically existing objects, such as metal signs, possessing measurable physical dimensions and locations. Physical markers are suitable for scenarios such as industrial inspection and traffic monitoring. For cases where multiple markers are physical entities, the method further includes the following steps: Obtain the displacement data corresponding to multiple markers; When the displacement data corresponding to multiple markers indicates that no displacement exists, the process of determining multiple candidate bounding boxes in the target image is allowed.

[0053] In the embodiments provided in this application, for physical markers, displacement data corresponding to multiple markers can be determined using sensors installed in their fixed-position bases or other positioning sensors. Before collecting lens offset detection data, it is first detected whether the markers have shifted, avoiding subjective or unintentional changes to the marker positions. This method has good adaptability to high-vibration environments in industrial settings, preventing subsequent detection failures. Displacement detection is a necessary condition for lens offset analysis, ensuring that the reference object has not been tampered with.

[0054] For example, accelerometers, tilt sensors, or microswitches are installed on the marker's fixed base to monitor physical displacement in real time. The absolute positional change of the marker can also be obtained through UWB (Ultra-Wideband), laser rangefinders, or visual marker tracking. If the displacement data of all markers are below the displacement threshold (e.g., translation <1mm, rotation <0.5°), it is determined that there is no displacement, allowing subsequent lens offset detection to proceed. If the displacement of any marker exceeds the limit, an alarm is triggered and detection is suspended until manual review or automatic reset.

[0055] For example, if some markers are slightly displaced (e.g., 1 out of 10 is shifted), the abnormal markers can be automatically removed, and only the stable markers are used to calculate the shift. The proportion of the aforementioned markers in all markers is less than a predetermined percentage.

[0056] According to the embodiments provided in this application, for some application scenarios that require flexible deployment of markers, virtual or temporarily generated reference objects can be used, such as projected images without a fixed physical form. When multiple markers are non-physical projections, before acquiring an image of the target scene with multiple markers, the method further includes: If the current pose of the projection device matches the predetermined pose, multiple markers are projected to their corresponding initial coordinates.

[0057] In the embodiments provided in this application, for scenarios requiring flexible deployment, such as temporary activities and dynamic environments, non-physical projected markers are used to achieve lens offset detection. The predetermined pose of the projection device can be determined through visual calibration or sensor fusion and stored as a reference. Before each projection, the current pose of the projection device is detected in real time by a camera or sensor and compared with the predetermined pose. After the pose verification is passed, multiple preset markers (such as points O, A, B, C, and D) are projected onto the surface of the target scene according to the initial coordinates. No physical installation of markers is required; the marker layout can be generated instantly on any surface. The projected surface in the target scene can be flat or uneven. Through the closed loop of projection device pose verification, dynamic marker generation, and visual detection, the high reliability of non-physical markers is achieved.

[0058] For example, when the marker is not a physical object, the detectability of the marker under strong / weak light conditions can be ensured by adjusting the projection brightness (e.g., increasing to 2000 lumens at night) or switching to infrared mode. Furthermore, for scenarios with lighting design, such as outdoor concerts, the system can automatically switch to infrared projection to overcome stage lighting interference and ensure marker detection rates.

[0059] According to the embodiments provided in this application, in order to facilitate subsequent automated calibration, it is necessary to identify what type of offset is being performed. For the offset of the acquisition viewpoint, it can include translation offset, rotation offset, and scaling offset.

[0060] Step S104: Among the multiple markers, there are predetermined markers used as offset baselines. Based on the current coordinates corresponding to the multiple markers in the target bounding box and the preset initial coordinates of the multiple markers, the acquisition offset result for image acquisition of the target scene is determined, which may include the following steps: Based on the current coordinates of the other markers (excluding the predetermined marker) and the current coordinates of the predetermined marker, determine the current vector pointing from the other markers to the predetermined marker; Based on the initial coordinates of other markers and the initial markings of the predetermined marker, determine the initial vector pointing from the other markers to the predetermined marker; Based on the deviation angle between the current vector and the initial vector, the acquired offset result is determined to be an offset with rotation type.

[0061] In the embodiments provided in this application, a predetermined marker is included among multiple markers as an offset baseline. Offset analysis is performed based on the current coordinates (i.e., the actual coordinates after image acquisition) and preset initial coordinates (i.e., the coordinates in the calibration state) of each marker in the target bounding box. For other markers besides the predetermined marker, a vector pointing from other markers to the predetermined marker is determined based on their current coordinates and the current coordinates of the predetermined marker, i.e., the current vector. Based on the initial coordinates of other markers and the initial coordinates of the predetermined marker, a vector pointing from other markers to the predetermined marker is determined, i.e., the initial vector. Through vector deviation angle analysis, it is possible to quantitatively determine whether there is a rotational offset in the acquisition viewpoint, such as a tilted shooting angle, providing a clear basis for the offset type for subsequent calibration. By comparing the deviation angle between the current vector and the initial vector, if an angle deviation exists, the acquisition offset result is determined to be a rotational type offset. Using the predetermined marker as the offset baseline reduces the accumulation of coordinate errors among multiple markers, making the offset calculation more benchmark-oriented and reliable.

[0062] For example, this embodiment addresses the automated detection of lens rotation offset by achieving accurate identification through a marker vector comparison method. For instance, one of multiple markers is designated as a reference point (e.g., point O), and the remaining markers (A, B, C, D) are used as detection points, forming a vector reference system with O as the origin. Based on the real-time detected coordinates, the vectors pointing from the other markers to the reference point are calculated, such as... = This gives us the current vector; similarly, we can obtain the initial vector using the initial coordinates.

[0063] For each pair of current vectors and initial vectors, such as and The deviation angle is calculated using the formula for the angle between the vectors of points O and A. :

[0064] By combining the angular changes of all vectors, the rotation offset is obtained. This rotation offset can be calculated by averaging and weighting the values. If the rotation offset indicates that the rotation angle exceeds an angle threshold, it is determined that the lens has a rotation offset, triggering a correction command.

[0065] According to the embodiments provided in this application, in step S104: the plurality of markers includes a predetermined marker used as an offset baseline. Based on the current coordinates corresponding to the plurality of markers in the target bounding box and the preset initial coordinates of the plurality of markers, the acquisition offset result for image acquisition of the target scene is determined, which may include the following steps: Determine the deviation range between the other markers (excluding the predetermined marker) and the predetermined marker from the set of markers; If the deviation magnitudes between multiple other markers and the predetermined markers match, the acquired offset result is determined to be an offset with scaling.

[0066] In the embodiments provided in this application, a predetermined marker is included among the multiple markers as an offset baseline. The offset is analyzed based on the current coordinates (i.e., actual acquisition coordinates) and preset initial coordinates (i.e., standard state coordinates) of each marker within the target bounding box. For other markers besides the predetermined marker, the coordinate deviation magnitude between them and the predetermined marker is calculated. The deviation magnitudes of multiple other markers and the predetermined marker are compared. If the deviation magnitudes are consistent, the acquisition offset result is determined to be a scaling type offset, that is, the image acquisition viewpoint is either magnified or reduced overall.

[0067] For example, one of the multiple markers is designated as the offset baseline (such as point O), and the remaining markers (A, B, C, etc.) are compared with marker O.

[0068] , ;

[0069]

[0070] An offset is considered to be of the scaling type when the following conditions are met: all marker δ values ​​are less than the set threshold; and the direction of distance change at each detection point is consistent, which means it is a proportional enlargement or reduction.

[0071] Based on the above embodiments and optional embodiments, this application also provides an optional implementation method. Figure 3 A schematic flowchart of the acquisition offset detection method according to an embodiment of this application is shown, as follows: Figure 3 As shown, step S1: After completing the camera installation and debugging, place markers and record the initial position coordinates of the key points of the markers in the lens monitoring image; Markers are reference objects used for camera lens shift detection. Example of a marker: Figure 2 As shown, Five key points for the marker, , , , The proportions of the four sides The actual size of the sign can be adjusted according to the specific scenario.

[0072] After the camera installation and debugging are completed, a suitable location needs to be selected within the camera lens's field of view to place a marker. The marker placement must meet two conditions: ① It must be clearly visible in the image captured by the lens and will not be obstructed during the monitoring period; ② The area of ​​the marker in the image captured by the lens must be proportional to the total area of ​​the image. ,in and Representing the edges of the marker and edge Length in the image, and These represent the length and width of the image, respectively.

[0073] After placing the marker, record five key points of the marker. The initial coordinate information in the image captured by the lens is represented as follows: , , , , ,in and Representing points respectively In the image shaft and The coordinates of the axes are similar, and so on for the rest.

[0074] Step S2: Extract the image of the current frame from the camera video stream at fixed intervals, and calculate the quality score of the current image using the image quality recognition model. If the quality score of the current image is greater than the set threshold, proceed to the next step S3 for processing and analysis; otherwise, repeat the current step S2. Under extreme conditions, such as severe weather and sudden changes in lighting, images captured by camera lenses may appear blurry. Using these low-quality images for camera lens shift detection analysis can easily lead to false positives. Therefore, image quality assessment is necessary before performing camera lens shift detection analysis. Here, the ResNet18 image classification network model is used for image quality assessment. The specific steps are as follows: Step S21: Extract the image of the current frame from the camera video stream at fixed intervals; Step S22: Perform preprocessing operations such as scaling and normalization on the current image to obtain the preprocessed image; Step S23: Input the preprocessed image into the trained ResNet18 image quality recognition network model and output the quality score QS (Quality Score) of the current image. The QS range is [0,1]. The larger the value, the higher the image quality, and vice versa. Step S24: If the quality score QS of the current image is greater than the set threshold, proceed to the next step S3 for subsequent camera lens offset detection and analysis; otherwise, re-extract the target image and repeat the image quality recognition.

[0075] Step S3: Based on the current image, further calculate the position coordinates of the key points of the marker in the current image using the marker localization model; In some extreme conditions, such as when the background features of the image have low recognizability, both manual visual inspection and feature comparison methods are prone to missed detections. Therefore, this paper adopts a landmark detection and localization method to provide a basis for judging camera lens offset detection. The landmark detection and localization task requires detecting the bounding rectangle of the landmark and the position coordinates of its five key points OABCD in the image, which is very similar to face keypoint detection and human pose estimation tasks. Here, the RetinaFace face keypoint detection model, which has undergone transfer learning, is used for landmark detection and localization. The specific steps are as follows: the target image is input into the trained RetinaFace landmark detection and localization model, and the output is the set of possible target bounding boxes (i.e., multiple candidate bounding boxes) of landmarks in the current image. ,in This indicates the total number of target borders to be output. Indicates the border number.

[0076]

[0077] in,( )and( ) respectively represent the first The coordinates of the top left and bottom right corners of the border in the image. Indicates the first The confidence level of each border, ( ), ( ), ( ), ( ), They represent the first Key points corresponding to each border Coordinates in the image, then from the bounding box set Select confidence level The largest border is taken as the border of the detected landmark in the current image, denoted as: .

[0078] Step S4: Calculate the deviation between the current position coordinates and the initial position coordinates of the key points of the marker to determine whether the camera lens has shifted.

[0079] The offset (i.e., the acquisition offset result) between the current position coordinates and the initial position coordinates of the key points of the markers is calculated. If the offset exceeds a set threshold, it is determined that the camera lens has shifted; otherwise, no shift has occurred. The distance between two points among multiple markers can be selected as a reference. For example, a predetermined multiple of the initial OA side length can be used as a reference to convert the total pixel offset into a relative proportion. The predetermined multiple is the magnification ratio, so that the offset falls within a more easily interpretable range.

[0080] Through the above processing, this application introduces an image quality recognition module into the camera lens offset detection process, which can effectively reduce false detections caused by severe weather and sudden changes in lighting. The method based on marker detection and localization, and bounding box confidence, enables automated detection of camera lens offset, effectively reducing missed detections caused by low background feature recognition in the monitored image. This allows for automated detection of camera lens offset in video surveillance systems, reducing labor costs and effectively improving the monitoring efficiency and automation level of the surveillance system. Combining the functional characteristics of the image quality recognition model and the marker detection and localization model, it exhibits good robustness, maintaining detection accuracy and real-time performance in various complex scenarios such as blurriness and noise, facilitating widespread adoption and demonstrating good practicality.

[0081] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide a device for acquiring offset detection. Figure 4 A block diagram of a data acquisition offset detection device according to an embodiment of this application is shown, including: The image acquisition module 401 is used to acquire images of a target scene with multiple markers to obtain a target image. The confidence determination module 402 is used to determine multiple candidate bounding boxes, each including multiple markers, in the target image, and the confidence scores corresponding to the multiple candidate bounding boxes respectively; the confidence scores are determined based on the relative positions of the multiple markers in the corresponding candidate bounding boxes. Bounding box determination module 403 is used to determine the target bounding box with the highest confidence among multiple candidate bounding boxes; The offset detection module 404 is used to determine the acquisition offset result of image acquisition of the target scene based on the current coordinates of multiple markers in the target bounding box and the preset initial coordinates of the multiple markers.

[0082] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0083] Figure 5 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 5 As shown, the electronic device includes a memory 501 and a processor 502. The memory 501 stores a computer program that can run on the processor 502. When the processor 502 executes the computer program, it implements the method described in the above embodiments. The number of memories 501 and processors 502 can be one or more. In a specific implementation, the electronic device may also include a communication interface 503 for communicating with external devices and exchanging data.

[0084] In practical implementation, if the memory 501, processor 502, and communication interface 503 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0085] Optionally, in a specific implementation, if the memory 501, processor 502 and communication interface 503 are integrated on a single chip, the memory 501, processor 502 and communication interface 503 can communicate with each other through an internal interface.

[0086] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0087] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in this application.

[0088] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0089] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0090] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0091] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0092] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0093] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0094] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0095] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0096] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0097] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0098] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0099] The above are merely exemplary embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for acquiring offset detection, characterized in that, include: Image acquisition is performed on a target scene with multiple markers to obtain a target image; In the target image, multiple candidate bounding boxes, each including the plurality of markers, are determined, along with confidence scores corresponding to each candidate bounding box; the confidence scores are determined based on the relative positions of the plurality of markers within the corresponding candidate bounding boxes. Determine the target bounding box with the highest confidence among multiple candidate bounding boxes; Based on the current coordinates of the multiple markers in the target bounding box and the preset initial coordinates of the multiple markers, the acquisition offset result for image acquisition of the target scene is determined.

2. The method according to claim 1, characterized in that, Before determining multiple candidate bounding boxes that respectively include the plurality of markers in the target image, and the confidence scores corresponding to the plurality of candidate bounding boxes, the method further includes: The image quality of the target image is determined by an image quality recognition model, which is trained based on historical images under different weather and lighting conditions. If the image quality is greater than a predetermined threshold, the process of determining the plurality of candidate bounding boxes in the target image is allowed to be performed.

3. The method according to claim 1, characterized in that, Determining the confidence level corresponding to each of the multiple candidate bounding boxes includes: The proportion of the markers is determined based on the area of ​​the bounding rectangle of the multiple markers and the image area of ​​the target image. The confidence level of each candidate bounding box is determined based on the proportion of the markers and their relative positions.

4. The method according to claim 1, characterized in that, The step of determining the acquisition offset result for image acquisition of the target scene based on the current coordinates of the multiple markers in the target bounding box and the preset initial coordinates of the multiple markers includes: Determine the coordinate deviation between the current coordinates and the initial coordinates corresponding to the plurality of markers; The pixel offset is determined based on the predetermined marker size; The acquisition offset result is determined based on the coordinate deviation and the pixel offset.

5. The method according to any one of claims 1 to 4, characterized in that, The plurality of markers are physical entities, and the method further includes: Obtain the displacement data corresponding to the multiple markers respectively; If the displacement data corresponding to the plurality of markers indicates that there is no displacement, the process of determining the plurality of candidate bounding boxes in the target image is permitted.

6. The method according to any one of claims 1 to 4, characterized in that, The multiple markers are non-physical projections. Before acquiring an image of the target scene with the multiple markers, the method further includes: If the current pose of the projection device matches the predetermined pose, the multiple markers are projected to the corresponding initial coordinates.

7. The method according to any one of claims 1 to 4, characterized in that, The plurality of markers includes predetermined markers used as offset baselines. The determination of the acquisition offset result for image acquisition of the target scene based on the current coordinates corresponding to each of the plurality of markers in the target bounding box and the preset initial coordinates of each of the plurality of markers includes: Based on the current coordinates of the other markers (excluding the predetermined marker) among the plurality of markers, and the current coordinates of the predetermined marker, determine the current vector pointing from the other markers to the predetermined marker; Based on the initial coordinates of the other markers and the initial mark of the predetermined marker, determine the initial vector pointing from the other markers to the predetermined marker; Based on the deviation angle between the current vector and the initial vector, the acquisition offset result is determined to be an offset with a rotation type.

8. The method according to any one of claims 1 to 4, characterized in that, The plurality of markers includes predetermined markers used as offset baselines. The determination of the acquisition offset result for image acquisition of the target scene based on the current coordinates corresponding to each of the plurality of markers in the target bounding box and the preset initial coordinates of each of the plurality of markers includes: Determine the deviation range between the other markers (excluding the predetermined marker) and the predetermined marker; If the deviation magnitudes between multiple other markers and the predetermined marker match, the acquired offset result is determined to be an offset with a scaling type.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

Images