Snapshot method, terminal device and computer readable storage medium
By predicting the capture quality and planning the target capture time in the future, this technology solves the problem that existing capture strategies cannot reasonably select the most valuable objects, and achieves high-quality image capture results in complex scenes.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-30
- Publication Date
- 2026-04-14
AI Technical Summary
Existing capture strategies cannot effectively select the most valuable tracking targets in complex scenes, resulting in low image clarity and poor capture performance. In particular, they are prone to missing the best shooting opportunity in scenes with multiple dense targets.
By acquiring the tracked object, the capture quality can be predicted for a future time period. Based on the capture quality, the capture object can be determined, and the target capture time can be planned. The high-speed camera can guide the high-resolution camera to capture, especially in multi-target clustering scenarios, to rationally select the most valuable capture object.
It improves the capture effect, reduces the capture of low-quality images, enhances resource utilization efficiency and capture control accuracy, and ensures the acquisition of high-quality images in complex scenes.
Smart Images
Figure CN121865102A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of image processing technology, and in particular relates to a snapshot method, terminal device and computer-readable storage medium. Background Technology
[0002] Road inspection refers to the detection of defects in roads and their ancillary facilities to promptly identify road defects, facility damage, and other issues, thereby ensuring traffic safety. During road inspections, vehicles (such as inspection cars or unmanned vehicles) are equipped with cameras that capture images of the road as the vehicle moves for inspection purposes.
[0003] Currently, vehicles operating these vehicles are equipped with two types of cameras: one is a high-frame-rate, low-resolution camera used to detect and track road defects across frames in real time; the other is a high-resolution camera (also known as a capture camera) used to acquire high-quality images of the target area at the "optimal moment." Existing capture strategies typically trigger captures with a fixed delay or fixed window, or continuously shoot at high frequencies. These methods are unsuitable for complex scenarios. Especially in scenarios with many dense targets, these methods cannot effectively select the most valuable tracking objects, easily missing the optimal shooting opportunity, resulting in low image clarity and poor capture performance. Summary of the Invention
[0004] This application provides a snapshot capture method, a terminal device, and a computer-readable storage medium, which can determine the "optimal timing" for snapshot capture, thereby effectively improving the snapshot capture effect.
[0005] In a first aspect, embodiments of this application provide a snapshot method, including: At least one tracking object is acquired; wherein the tracking object is an object tracked based on an image captured by a first camera on a mobile device; Predict the first capture quality for each tracked object within a first future time period; The capture object is determined from the at least one tracked object based on the first capture quality of each of the tracked objects; The target capture time for the target object is planned based on the moving speed of the mobile device; The second camera on the mobile device is controlled to capture the target image at the specified capture time; wherein the frame rate of the second camera is lower than that of the first camera.
[0006] In this embodiment, the first camera is equivalent to a high-speed camera, and the second camera is equivalent to a snapshot camera. The snapshot quality of the tracked object detected by the high-speed camera in the future is evaluated in real time, and the snapshot object is determined based on the snapshot quality. Especially in the scenario of multiple targets, the above method can reasonably select the most valuable snapshot object and give the optimal snapshot time (i.e. target snapshot time) for the snapshot object, which greatly improves the snapshot effect.
[0007] In one implementation, predicting the first capture quality of each tracked object within a first future time period includes: Predict the quality score of the captured image corresponding to each predicted moment within the first future time period for the tracked object; The first capture quality of the tracked object within the first future time period is determined based on the quality score corresponding to each predicted time.
[0008] The above scheme can predict the capture quality of images in the future, so that the subsequent capture control can adapt to the state of the capture object in the future time period and reduce the occurrence of low-quality images. In addition, the future time period is subdivided into various prediction times, which is equivalent to refining the time granularity and improving the accuracy of subsequent capture control.
[0009] In one implementation, predicting the quality score of the captured image corresponding to each prediction time of the tracked object within the first future time period includes: Obtain the actual location of the currently tracked object on the ground; The pixel trajectory of the tracked object within the first future time period is inferred based on the actual location; wherein, the pixel trajectory includes the pixel position in the captured image corresponding to each prediction time. The quality score of the captured image corresponding to each predicted moment within the first future time period is calculated based on the pixel trajectory.
[0010] In the above scheme, by calculating the actual position-pixel trajectory-quality score, the motion in the physical space is correlated with the presentation in the image space. The physical motion of the tracked object is converted into a quantitative analysis of the image dimension, so that the subsequent capture control process can accurately map the predicted presentation in the image space (i.e. capture quality) to the physical space to determine the precise capture time, thereby achieving precise control of high-quality capture.
[0011] In one implementation, determining the capture object from the at least one tracked object based on a first capture quality of each tracked object includes: The priority of each tracked object is calculated based on the first capture quality of each tracked object; The capture object is determined from the at least one tracking object based on the priority of each tracking object.
[0012] In the above scheme, determining the capture target based on the capture quality ensures that high-quality capture images are subsequently obtained, thus improving the capture effect. Furthermore, when multiple objects are being tracked, this method prioritizes capturing objects with higher capture quality, avoiding wasting resources on low-quality objects and improving resource utilization efficiency.
[0013] In one implementation, the step of planning the target capture time for the target object based on the moving speed of the mobile device includes: The second capture quality of the captured object is obtained; wherein, the second capture quality is the capture quality of the captured image of the captured object obtained when the mobile device moves at the current moving speed, and the second capture quality is determined based on the quality score of the captured image corresponding to each predicted time in the second future time period; The target capture time of the object is determined based on the predicted time corresponding to the second capture quality. Before the target capture time, the first capture time for the target is replanned according to the moving speed of the mobile device at a preset period; If the first capture time is different from the target capture time, then the target capture time is updated according to the first capture time.
[0014] In the above solution, the capture time is planned according to the moving speed of the mobile device, so that the capture control can adapt to the mobility of the device, thereby ensuring the stability of the capture quality. In addition, through periodic replanning, the capture time is fine-tuned when a better plan is found, so as to balance continuous optimization and avoid frequent jitter, thus achieving more accurate and better capture control.
[0015] In one implementation, the step of replanning the first capture time for the target object according to the moving speed of the mobile device at a preset period includes: Obtain at least one candidate speed of the mobile device; Predict the third capture quality corresponding to each candidate speed; wherein, the third capture quality is the capture quality of the capture image of the capture object obtained when the mobile device moves at the candidate speed; The target vehicle speed is determined from the at least one candidate speed based on the third capture quality corresponding to each candidate speed; The first capture time for the target object is planned based on the target vehicle speed.
[0016] The above scheme is equivalent to calculating the quality of the captured image at several possible vehicle speeds, selecting the target vehicle speed with higher capture quality, and further improving the control effect and adaptability of the capture control by shifting from passive reception quality to active control quality.
[0017] In one implementation, after acquiring at least one tracked object, the method further includes: A first capture request is generated when a preset condition is met; wherein, the first capture request is used to trigger the second camera to capture; the preset condition is that any pixel position of the tracked object in the image captured by the first camera intersects with a preset detection box; The second camera is controlled to capture images according to the first capture request.
[0018] In the above scheme, when the tracked object in the captured image touches the boundary of the preset detection box, a "last-ditch capture" is triggered. Even if the planning fails, the captured image can still be obtained, which helps to improve the stability of the system.
[0019] In one embodiment, the method further includes: If the first capture request and the second capture request exist simultaneously, it is determined whether the tracking object corresponding to the first capture request is the capture object; wherein, the second capture request is used to trigger the second camera to capture at the target capture time; If the tracking object corresponding to the first capture request is the capture object, then the first capture request is deleted.
[0020] In the above solution, when multiple capture requests for the same target exist simultaneously, one capture request is selected and retained, which reduces interference and duplication from multiple requests, thereby reducing resource waste and improving resource utilization.
[0021] Secondly, embodiments of this application provide a snapshot device, including: An acquisition module is used to acquire at least one tracked object; wherein the tracked object is an object tracked based on an image captured by a first camera on a mobile device; The prediction module is used to predict the first capture quality of each tracked object within a first future time period; The selection module is used to determine the capture object from the at least one tracking object based on the first capture quality of each tracking object; The planning module is used to plan the target capture time for the target object based on the moving speed of the mobile device; The control module is used to control the second camera on the mobile device to capture the target based on the target capture time; wherein the frame rate of the second camera is lower than the frame rate of the first camera.
[0022] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the snapshot method as described in any one of the first aspects above.
[0023] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the snapshot method as described in any one of the first aspects above.
[0024] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the snapshot method described in any one of the first aspects.
[0025] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description
[0026] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 This is a flowchart illustrating a snapshot method provided in one embodiment of this application; Figure 2 This is a flowchart illustrating a snapshot method provided in another embodiment of this application; Figure 3 This is a structural block diagram of a snapshot device provided in one embodiment of this application; Figure 4 This is a schematic diagram of the structure of a terminal device provided in one embodiment of this application. Detailed Implementation
[0028] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0029] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.
[0030] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0031] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."
[0032] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0033] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.
[0034] Road inspection refers to the detection of defects in roads and their ancillary facilities to promptly identify road defects, facility damage, and other issues, thereby ensuring traffic safety. During road inspections, vehicles (such as inspection cars or unmanned vehicles) are equipped with cameras that capture images of the road as the vehicle moves for inspection purposes.
[0035] Currently, vehicles operating these vehicles are equipped with two types of cameras: one is a high-frame-rate, low-resolution camera used to detect and track road defects across frames in real time; the other is a high-resolution camera (also known as a capture camera) used to acquire high-quality images of the target area at the "optimal moment." Existing capture strategies typically trigger captures with a fixed delay or fixed window, or continuously shoot at high frequencies. These methods are unsuitable for complex scenarios. Especially in scenarios with many dense targets, these methods cannot effectively select the most valuable tracking objects, easily missing the optimal shooting opportunity, resulting in low image clarity and poor capture performance.
[0036] Based on this, this application provides a snapshot method. In this application, the snapshot quality of the tracked object detected by the high-speed camera in the future time is evaluated in real time, and the snapshot object is determined based on the snapshot quality. Especially in scenarios with multiple targets clustered together, the above method can reasonably select the most valuable snapshot object and give the optimal snapshot time (i.e., the target snapshot time) for the snapshot object, which greatly improves the snapshot effect.
[0037] The image capture method of this application can be applied to various fields, including but not limited to: road inspection, security monitoring, traffic management, sports event recording, wildlife observation, and other scenarios that require accurate capture of dynamic targets to obtain high-quality images. For example, in security monitoring, suspicious persons or abnormal behaviors appearing in the monitored area can be tracked, their future movement trajectory and posture changes can be predicted, and high-definition cameras can be triggered to capture images at the optimal moment, providing strong support for subsequent event analysis and evidence preservation; in sports event recording, the key actions of athletes (such as shooting, jumping, etc.) can be accurately predicted and captured, capturing the most dynamic and expressive moments.
[0038] See Figure 1 This is a flowchart illustrating the image capture method provided in this application embodiment. It is intended as an example and not a limitation. The method may include the following steps: S101, acquire at least one tracked object.
[0039] In this embodiment, the mobile device is equipped with a first camera and a second camera, wherein the frame rate of the first camera is higher than that of the second camera. It is understood that the first camera is a high-frame-rate, low-resolution high-speed tracking camera used to detect and track road surface defects across frames in real time; the second camera is a high-resolution snapshot camera used to acquire high-quality images of the target area at the "optimal moment".
[0040] The tracked object is an object tracked based on images captured by the first camera on the mobile device. For example, objects are detected based on images captured by the first camera, and the detected objects are correlated across frames to obtain stable tracking IDs. Each tracking ID corresponds to one tracked object.
[0041] The snapshot method provided in this application can be applied to various scenarios. For example, in a road inspection application scenario, the mobile device can be a vehicle / autonomous vehicle, and the tracked object can be a road surface defect. In a tracking and shooting application scenario, the mobile device can be a movable shooting device, and the tracked object can be a person.
[0042] S102, predicts the first capture quality of each tracked object within the first future time period.
[0043] The first future time period refers to a time interval following the current moment. Optionally, the first future time period is a time interval that satisfies the following conditions: 1. The projection point is within the field of view and the depth is valid; 2. Considering the software trigger delay and exposure time, it is still possible to complete the capture within the time slot; 3. Camera resources have available time slots during this period (no overlapping exposures). In one implementation, S102 includes: Predict the quality score of the captured image corresponding to each predicted moment within the first future time period for the tracked object; The first capture quality of the tracked object within the first future time period is determined based on the quality score corresponding to each predicted time.
[0044] Optionally, the highest quality score of the tracked object within a first future time period can be determined as the first capture quality of the tracked object.
[0045] Optionally, the average of all quality scores for the tracked object within a first future time period can be calculated, and then the quality score that is closest to the average value can be selected from all quality scores within the first future time period as the first capture quality of the tracked object.
[0046] The above scheme can predict the capture quality of images in the future, so that the subsequent capture control can adapt to the state of the capture object in the future time period and reduce the occurrence of low-quality images. In addition, the future time period is subdivided into various prediction times, which is equivalent to refining the time granularity and improving the accuracy of subsequent capture control.
[0047] In one implementation, the method for predicting the quality score at each prediction time includes: Obtain the actual location of the currently tracked object on the ground; The pixel trajectory of the tracked object within the first future time period is inferred based on its actual location; wherein, the pixel trajectory includes the pixel position in the captured image corresponding to each prediction time. The quality score of the captured image is calculated based on the pixel trajectory for each predicted moment within the first future time period.
[0048] Optionally, the tracked object in the image can be mapped to the world coordinate system of the ground based on camera parameters (such as intrinsic and extrinsic parameters) to obtain the actual position of the tracked object on the ground. Specifically, the camera's pixel observations are back-projected into a three-dimensional line-of-sight direction and intersected with a pre-calibrated road surface plane to obtain an estimate of the tracked object's three-dimensional position on the ground, i.e., its actual position. In this method, it is equivalent to treating the tracked object in the image as "the direction the camera sees it," and "hitting the ground" along this line-of-sight direction; the landing point is the actual position of the tracked object on the ground.
[0049] For example, for the center pixel of the detection box in camera d at time t0, which is u d First, the center pixel u d Back projection is the direction of the line of sight. In the formula [u d [1] represents homogeneous coordinates. K d [BJ1] It's the camera's internal parameters. R d (t0) represents the position and orientation of the camera at time t0 [2] (e.g., rotation matrix); then the line of sight is calculated. r d The intersection with the pre-calibrated ground is In the formula, G represents the actual position of the tracked object on the ground (world coordinates). C d (t0) represents the position and orientation of the camera at time t0 (e.g., rotation matrix) [3]. n and d These are the normal and offset in the plane equation of the pre-calibrated ground, respectively.
[0050] Optionally, sensor data such as Visual-Inertial Odometry (VIO) and / or Global Navigation Satellite System (GNSS) can be used to extrapolate the pose deviation between every two adjacent predicted times within the first future time period; then, starting from the actual position of the tracked object, the actual movement trajectory of the tracked object within the first future time period can be extrapolated based on the pose deviation between every two predicted times; finally, the actual position corresponding to each predicted time in the actual movement trajectory can be transformed into a pixel coordinate system to obtain the pixel position of the tracked object at each predicted time.
[0051] For example, at time t, the pose (C) of the camera is captured using VIO and GNSS. s ( t ), R s ( t Projecting ground point G onto the image plane of the capture camera: x s ( t )= K s ( t ) R s (G- C s ( t Then, the pixel coordinates are normalized to obtain: The validity condition is that z > 0 and the pixel coordinates are... u s ( t It is located within the image area (including safety margins).
[0052] Optionally, for each prediction time, the index score corresponding to that prediction time is evaluated according to multiple quality indicators; the index scores corresponding to each of the multiple quality indicators are weighted and summed to obtain the quality score corresponding to that prediction time.
[0053] For example, quality metrics can include resolution metrics, sharpness metrics, and geometric feature metrics.
[0054] The resolution index score can be calculated using the formula... Calculate; where, cov px ( t [4] represents the pixel coverage of the tracked object in the image, and L represents the expected actual size of the tracked target. N min Let GSD be the minimum pixel coverage threshold for the tracked object, and GSD be the ground sampling distance. If the above inequality is not satisfied, then... N min Reduce the weight (e.g., set it to 0).
[0055] GSD (Gross Surface Area Code) represents the number of pixels that represent the actual distance traveled on the ground in an image. A smaller GSD indicates more pixels of movement, meaning the ground is more "magnified" in the image, and details are clearer. GSD can be calculated using the formula... Calculation. Where δ is the preset increment. t1 and t2 are two orthogonal unit tangents on the ground, and G is the actual location point on the ground. Understandably, resolution metrics are used to assess whether the area of pixels covered by the tracked object in an image is "large enough".
[0056] The resolution index score can be calculated using the formula. Calculate. In the formula, b max The upper limit of the maximum allowed motion blur; v px [5] is the pixel velocity of the object on the image plane of the camera. Δ t This is the time step. If the indicator score cannot satisfy the above inequality, the gating can be adjusted. b max .
[0057] Understandably, the sharpness metric is used to evaluate the motion blur of an image, that is, how fast the tracked object moves in the image. The faster it moves, the higher the degree of motion blur, and the easier it is to capture a blurry image.
[0058] Geometric feature indices can be derived from formulas Calculate. In the formula, cosθ observation vector v ( t )= G- C s ( t ) and ground normal n The cosine of the included angle, when cosθ When the pixel size is small, image quality degrades and needs to be suppressed in geometric scoring. u s ( t ) and image center u ctr The normalized distance is ; m This is a safe distance from the image boundary to avoid capturing images too close to the edge. This is a limiting function; α To control the weight of "frontal priority" and "center priority".
[0059] Understandably, geometric feature metrics are used to assess how close the tracked object is to the center of the frame. The closer the tracked object is to the center of the frame, the more stable it is, the less distortion it has, and the less likely it is to run out of the frame due to camera shake; therefore, "shooting from the center" should be encouraged. At the same time, the shooting angle should also be considered: it should be as directly facing the target as possible, and as centered as possible.
[0060] Optionally, it can be based on the formula Calculate the quality score of the captured image at each prediction time. In the formula, the score for the resolution index is... , Nmin The minimum pixel coverage threshold, N good The target coverage is rated as "good"; the clarity indicator score is... The number of blurred pixels predicted is , b good To determine the preferred threshold, b max This is the upper limit of tolerance. The index score for the geometric feature index is... Uncertainty penalty U pose ∈[0,1] represents the deduction term obtained after normalization of the combined attitude prediction covariance and target localization uncertainty. w r , w b , w g and w u As the weight, satisfying w r + w b + w g + w u =1.
[0061] Optionally, gating conditions can be set; if any of the following conditions are not met, then... Q ( t )=0: A. The projection point is outside the visible field of view (z≤0 or u s ( t )) B. Pixel coverage is below the minimum threshold ( cov px [6]( t )< N min ) C. Exposure time constraints are not feasible.
[0062] In the above scheme, by calculating the actual position-pixel trajectory-quality score, the motion in the physical space is correlated with the presentation in the image space. The physical motion of the tracked object is converted into a quantitative analysis of the image dimension, so that the subsequent capture control process can accurately map the predicted presentation in the image space (i.e. capture quality) to the physical space to determine the precise capture time, thereby achieving precise control of high-quality capture.
[0063] S103, determine the capture object from at least one tracked object based on the first capture quality of each tracked object.
[0064] In one implementation, S103 includes: The priority of each tracked object is calculated based on the first capture quality of each tracked object. The target to be captured is determined from at least one tracked object based on the priority of each tracked object.
[0065] Optionally, priority can be determined solely based on capture quality. Specifically, the tracking object corresponding to the maximum value among the calculated first capture quality values is determined as the capture object. This can be understood as the first capture quality reflecting, under the constraints of current geometric relationships, vehicle speed, and camera parameters, the "theoretically best imaging quality achievable" for the tracking object within a future time period. Using the first capture quality as a priority factor directly drives the system to allocate limited capture resources to targets more likely to yield clear, well-covered, and reasonably oriented evidence images, thereby improving overall data quality and usability.
[0066] Optionally, priorities can be determined based on capture quality and other factors. For example, other factors may include improvement level and urgency. Improvement level is used to assess the value of re-capturing a tracked object, while urgency is used to assess the timeliness of performing the capture.
[0067] For example, according to the formula Calculate the degree of improvement. Where, Q i,prev For the i-th tracked object, the best quality previously captured, Let Δ represent the quality of the first capture of the i-th tracked object in this instance. It can be understood that Δ... Q i This is used to measure the value of "re-capturing". When ambient lighting, mobile device attitude / speed, occlusion, etc., change, the same tracked object may yield higher quality images at later times; this is achieved by introducing Δ. Q i The system will neither permanently eliminate the defect simply because it has been photographed before, nor will it repeatedly capture the same target without improving quality. This creates an interpretable constraint between "repeatable attention" and "avoiding redundant resource consumption".
[0068] For example, according to the formula Calculate the urgency. In the formula, L i = d i - t now - p i The remaining relaxation time, d iThis refers to the cutoff time (e.g., the end of the first future time period). p i To account for processing costs (such as exposure time and software / hardware overhead). t now The current moment; L i When it approaches 0 H i A value close to 1 indicates high urgency. H i The larger the value, the more urgent the process. When there's only a short time left before the window closes, the system will be more inclined to process it first, even if its theoretically optimal quality isn't the highest; parameters α Decide "how quickly things start to get urgent" (the rate at which the urgency increases). Understandably, H i This factor is used to explicitly express the risk of "it will be too late if not handled now" in multi-objective competition, preventing the system from missing tracking objects near the window while pursuing higher or larger targets. This factor essentially incorporates deadline information into priority, making scheduling more aligned with the real-time requirements of online systems.
[0069] Optionally, the priority of the tracking object can be obtained by weighting and summing the first capture quality, improvement degree, and urgency; then, the tracking object with the highest priority is determined as the capture object. It can be understood that evaluating priority based on the above three priority factors is equivalent to simultaneously considering the maximum image quality that can be captured under current conditions, whether the image quality of this capture is significantly improved compared to historical captures, and whether there is a risk of running out of time to capture the image. These factors collectively determine the selection and scheduling order of capture objects.
[0070] In practical applications, regarding the weight of improvement, if the weight is too large, the system may overly favor "old defects that can be improved," thus crowding out opportunities to capture new defects; if the weight is too small, the system will have difficulty timely revisiting and obtaining better evidence when the environment improves. In practice, this can be addressed through weight adjustment and other methods. Q i,prev Recording strategies (such as taking the best historical data or introducing time decay / scene tags) are used to control the frequency of revisits and resource consumption.
[0071] Regarding urgency, excessive urgency forces the system to rush through the window, potentially resulting in images of mediocre quality; insufficient urgency increases the probability of missed shots. Choosing the appropriate urgency... α With weights, the strategy can be adjusted between "trying not to miss any shots" and "trying to capture the best shots"; at the same time, in implementation, mechanisms such as minimum quality thresholds and fallback captures (safety frames) can be combined to reduce the risk in extreme cases.
[0072] In the above scheme, determining the capture target based on the capture quality ensures that high-quality capture images are subsequently obtained, thus improving the capture effect. Furthermore, when multiple objects are being tracked, this method prioritizes capturing objects with higher capture quality, avoiding wasting resources on low-quality objects and improving resource utilization efficiency.
[0073] S104, plans the target capture time for the subject based on the moving speed of the mobile device.
[0074] In one implementation, S104 includes: The second capture quality of the captured object is obtained; wherein, the second capture quality is the capture quality of the captured image of the captured object obtained when the mobile device moves at the current moving speed, and the second capture quality is determined based on the quality score of the captured image corresponding to each predicted time in the second future time period; The target capture time of the target object is determined based on the predicted time corresponding to the second capture quality. Before the target capture time, the first capture time for the target object is re-planned according to the moving speed of the mobile device at a preset period; If the first capture time is different from the target capture time, the target capture time is updated based on the first capture time.
[0075] The second future time period can be the same as the first future time period. In this case, the first capture quality of the captured object calculated above can be directly sampled as the second capture quality. Alternatively, the second future time period can be different from the first future time period. In this case, the second capture quality within the second future time period can be calculated using the same method as the first capture quality. It is understood that if the second future time period is different from the first future time period, the second future time period is shorter than the first future time period.
[0076] In the above solution, the capture time is planned according to the moving speed of the mobile device, so that the capture control can adapt to the mobility of the device, thereby ensuring the stability of the capture quality. In addition, through periodic replanning, the capture time is fine-tuned when a better plan is found, so as to balance continuous optimization and avoid frequent jitter, thus achieving more accurate and better capture control.
[0077] In one implementation, the step of rescheduling the first capture time for the subject includes: Obtain at least one candidate speed from the mobile device; Predict the third capture quality corresponding to each candidate speed; wherein, the third capture quality is the capture quality of the capture image of the capture object obtained when the mobile device moves at the candidate speed; The target vehicle speed is determined from at least one candidate speed based on the third capture quality corresponding to each candidate speed. The first capture time for the target vehicle is planned based on the target vehicle speed.
[0078] The calculation method for the third capture quality corresponding to each candidate speed is the same as that for the first capture quality. For details, please refer to the calculation method for the first capture quality in S102, which will not be repeated here.
[0079] Optionally, the target speed can be determined by selecting the candidate speed that best matches the current speed and meets the third capture quality threshold. If no third capture quality meets the quality threshold, the candidate speed corresponding to the maximum value among the calculated third capture qualities can be determined as the target speed, or the current speed can be used as the target speed.
[0080] The above scheme is equivalent to calculating the quality of the captured image at several possible vehicle speeds, selecting the target vehicle speed with higher capture quality, and further improving the control effect and adaptability of the capture control by shifting from passive reception quality to active control quality.
[0081] S105 controls the second camera on the mobile device to capture the target based on the target capture time.
[0082] In one implementation, S105 includes: A second capture request is generated based on the target capture time; wherein, the second capture request is used to trigger the second camera to capture at the target capture time; the second capture request is sent to the second camera to instruct the second camera to capture.
[0083] Understandably, the second capture request is a timed request, meaning that the capture is only triggered at a specific time.
[0084] Figure 1 In the described embodiment, the first camera is equivalent to a high-speed camera, and the second camera is equivalent to a snapshot camera. The snapshot quality of the tracked object detected by the high-speed camera within a future timeframe is evaluated in real time. Based on the snapshot quality, the target object is determined, essentially using the high-speed camera to guide the snapshot camera's action, reducing missed or false snapshots caused by blind or fixed snapshots. Especially in scenarios with multiple targets, this method can rationally select the most valuable target and provide the optimal snapshot time (i.e., the target snapshot moment) for that target, significantly improving the snapshot effect.
[0085] In one implementation, after acquiring at least one tracked object, the method further includes: A first capture request is generated when a preset condition is met; the second camera is then controlled to capture an image based on the first capture request. The first capture request triggers the second camera to capture an image; the preset condition is that the pixel position of any tracked object in the image captured by the first camera intersects with a preset detection box.
[0086] Understandably, the first capture request is equivalent to a "as soon as possible" request, meaning it triggers the capture immediately if there are no resource conflicts. The difference between the first and second capture requests is that the first capture request has no specific target time and focuses more on the timeliness of the capture.
[0087] In the above scheme, when the tracked object in the captured image touches the boundary of the preset detection box, a "last-ditch capture" is triggered. Even if the planning fails, the captured image can still be obtained, which helps to improve the stability of the system.
[0088] In one embodiment, the method further includes: If both the first capture request and the second capture request exist, then determine whether the tracking object corresponding to the first capture request is a capture object; If the tracking object corresponding to the first capture request is a capture object, then delete the first capture request.
[0089] In the above solution, when multiple capture requests for the same target exist simultaneously, one capture request is selected and retained, which reduces interference and duplication from multiple requests, thereby reducing resource waste and improving resource utilization.
[0090] For example, see Figure 2 This is a schematic diagram of the overall process of the capture control provided in the embodiments of this application. It is intended as an example and not a limitation. Figure 2 As shown, the overall process of snapshot control may include the following steps: S201, After acquiring at least one tracked object, evaluate the first capture quality of each tracked object.
[0091] S202, determine the capture object from at least one tracked object based on the first capture quality of each tracked object.
[0092] S203, based on the current moving speed of the mobile device, initially plan the target capture time for the target object.
[0093] S204, Generate a second capture request based on the target capture time.
[0094] The implementation principles of S201-S204 are the same as those of S101-S104. For details, please refer to the description in the embodiments of S101-S104, which will not be repeated here.
[0095] S205 determines whether the subject has been missed or has already been captured.
[0096] If the target has been missed, or if the target has already been captured, then execute S201 to select the next target.
[0097] Understandably, if the actual location of the current mobile device is misaligned with the actual location of the target object, it is determined that the target object has been missed. If a second capture request has been executed, it is determined that the target object has been captured.
[0098] S206 If the target object is not missed, but the target object is not captured, the target capture time of the target object is re-planned according to the preset cycle.
[0099] S207 updates the second capture request based on the re-planned target capture time.
[0100] S207 can be found in the embodiment of S104 which re-plans the first capture time and updates the target capture time based on the first capture time, and will not be described again here.
[0101] In some implementations, the target speed corresponding to the updated target capture time can be displayed to the user through the user interface, so that the user can control the movement of the mobile device according to the target speed.
[0102] S208, conduct a fallback test.
[0103] S209, when the preset conditions are met, a first capture request is generated.
[0104] Steps S208-S209 can be found in the description of the above embodiment for generating the first capture request, and will not be repeated here.
[0105] It is understandable that the branches in steps S201-S204 and steps S208-S209 can be processed in parallel without interference. Resource scheduling is performed only when both the first and second capture requests exist. Specifically, it is determined whether the tracking object corresponding to the first capture request is a capture object; if the tracking object corresponding to the first capture request is a capture object, the first capture request is deleted.
[0106] In this embodiment, the capture quality of the tracked object detected by the high-speed camera in the future is evaluated in real time. The capture object is determined based on the capture quality, which is equivalent to using the high-speed camera to guide the capture camera's capture action, reducing missed or false captures caused by blind or fixed capture. Especially in scenarios with multiple targets, the above method can reasonably select the most valuable capture object and provide the optimal capture time (i.e., the target capture time) for that object, greatly improving the capture effect. In addition, parallel fallback detection ensures that capture images can still be obtained even if planning fails, which helps improve the stability of the system.
[0107] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0108] Corresponding to the image capture method described in the above embodiments, Figure 3 This is a structural block diagram of the image capture device provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.
[0109] Reference Figure 3 The image capture device 3 provided in this application embodiment includes: The acquisition module 31 is used to acquire at least one tracked object; wherein the tracked object is an object tracked based on an image captured by a first camera on a mobile device. The prediction module 32 is used to predict the first capture quality of each tracked object within a first future time period.
[0110] Selection module 33 is used to determine the capture object from the at least one tracked object based on the first capture quality of each tracked object. The planning module 34 is used to plan the target capture time for the target object based on the moving speed of the mobile device.
[0111] The control module 35 is used to control the second camera on the mobile device to capture the target according to the target capture time; wherein the frame rate of the second camera is lower than the frame rate of the first camera.
[0112] Optionally, the prediction module 32 is also used for: Predict the quality score of the captured image corresponding to each predicted moment within the first future time period for the tracked object; The first capture quality of the tracked object within the first future time period is determined based on the quality score corresponding to each predicted time.
[0113] Optionally, the prediction module 32 is also used for: Obtain the actual location of the currently tracked object on the ground; The pixel trajectory of the tracked object within the first future time period is inferred based on the actual location; wherein, the pixel trajectory includes the pixel position in the captured image corresponding to each prediction time. The quality score of the captured image corresponding to each predicted moment within the first future time period is calculated based on the pixel trajectory.
[0114] Optionally, selection module 33 is also used for: The priority of each tracked object is calculated based on the first capture quality of each tracked object; The capture object is determined from the at least one tracking object based on the priority of each tracking object.
[0115] Optionally, the planning module 34 is also used for: The second capture quality of the captured object is obtained; wherein, the second capture quality is the capture quality of the captured image of the captured object obtained when the mobile device moves at the current moving speed, and the second capture quality is determined based on the quality score of the captured image corresponding to each predicted time in the second future time period; The target capture time of the object is determined based on the predicted time corresponding to the second capture quality. Before the target capture time, the first capture time for the target is replanned according to the moving speed of the mobile device at a preset period; If the first capture time is different from the target capture time, then the target capture time is updated according to the first capture time.
[0116] Optionally, the planning module 34 is also used for: Obtain at least one candidate speed of the mobile device; Predict the third capture quality corresponding to each candidate speed; wherein, the third capture quality is the capture quality of the capture image of the capture object obtained when the mobile device moves at the candidate speed; The target vehicle speed is determined from the at least one candidate speed based on the third capture quality corresponding to each candidate speed; The first capture time for the target object is planned based on the target vehicle speed.
[0117] Optionally, the control module 35 is also used for: After acquiring at least one tracked object, a first capture request is generated when a preset condition is met; wherein, the first capture request is used to trigger the second camera to capture; the preset condition is that any pixel position of the tracked object in the image captured by the first camera intersects with a preset detection box; The second camera is controlled to capture images according to the first capture request.
[0118] Optionally, the control module 35 is also used for: If the first capture request and the second capture request exist simultaneously, it is determined whether the tracking object corresponding to the first capture request is the capture object; wherein, the second capture request is used to trigger the second camera to capture at the target capture time; If the tracking object corresponding to the first capture request is the capture object, then the first capture request is deleted.
[0119] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.
[0120] in addition, Figure 3 The image capture device shown can be a software unit, a hardware unit, or a combination of software and hardware built into an existing terminal device. It can also be integrated into the terminal device as a separate accessory, or it can exist as a standalone terminal device.
[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0122] Figure 4 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. For example... Figure 4 As shown, the terminal device 4 in this embodiment includes: at least one processor 40 ( Figure 4(Only one is shown in the image) a processor, a memory 41, and a computer program 42 stored in the memory 41 and executable on the at least one processor 40, wherein the processor 40 executes the computer program 42 to implement the steps in any of the above-described snapshot method embodiments.
[0123] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4 This is merely an example of terminal device 4 and does not constitute a limitation on terminal device 4. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.
[0124] The processor 40 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0125] In some embodiments, the memory 41 may be an internal storage unit of the terminal device 4, such as a hard disk or memory of the terminal device 4. In other embodiments, the memory 41 may be an external storage device of the terminal device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device 4. Furthermore, the memory 41 may include both internal and external storage units of the terminal device 4. The memory 41 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 41 can also be used to temporarily store data that has been output or will be output.
[0126] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the above-described method embodiments.
[0127] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments.
[0128] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.
[0129] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0130] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0131] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0132] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0133] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method for capturing images, characterized in that, The capture method includes: At least one tracking object is acquired; wherein the tracking object is an object tracked based on an image captured by a first camera on a mobile device; Predict the first capture quality for each tracked object within a first future time period; The capture object is determined from the at least one tracked object based on the first capture quality of each of the tracked objects; The target capture time for the target object is planned based on the moving speed of the mobile device; The second camera on the mobile device is controlled to capture the target image at the specified capture time; wherein the frame rate of the second camera is lower than that of the first camera.
2. The image capture method as described in claim 1, characterized in that, The prediction of the first capture quality of each tracked object within the first future time period includes: Predict the quality score of the captured image corresponding to each predicted moment within the first future time period for the tracked object; The first capture quality of the tracked object within the first future time period is determined based on the quality score corresponding to each predicted time.
3. The snapshot method as described in claim 2, characterized in that, The prediction of the quality score of the captured image corresponding to each prediction time of the tracked object within the first future time period includes: Obtain the actual location of the currently tracked object on the ground; The pixel trajectory of the tracked object within the first future time period is inferred based on the actual location; wherein, the pixel trajectory includes the pixel position in the captured image corresponding to each prediction time. The quality score of the captured image corresponding to each predicted moment within the first future time period is calculated based on the pixel trajectory.
4. The snapshot method as described in claim 1, characterized in that, The step of determining the capture object from the at least one tracked object based on the first capture quality of each tracked object includes: The priority of each tracked object is calculated based on the first capture quality of each tracked object; The capture object is determined from the at least one tracking object based on the priority of each tracking object.
5. The snapshot method as described in claim 1, characterized in that, The step of planning the target capture time for the target object based on the moving speed of the mobile device includes: The second capture quality of the captured object is obtained; wherein, the second capture quality is the capture quality of the captured image of the captured object obtained when the mobile device moves at the current moving speed, and the second capture quality is determined based on the quality score of the captured image corresponding to each predicted time in the second future time period; The target capture time of the object is determined based on the predicted time corresponding to the second capture quality. Before the target capture time, the first capture time for the target is replanned according to the moving speed of the mobile device at a preset period; If the first capture time is different from the target capture time, then the target capture time is updated according to the first capture time.
6. The snapshot method as described in claim 5, characterized in that, The step of replanning the first capture time for the target object according to the moving speed of the mobile device at a preset period includes: Obtain at least one candidate speed of the mobile device; Predict the third capture quality corresponding to each candidate speed; wherein, the third capture quality is the capture quality of the capture image of the capture object obtained when the mobile device moves at the candidate speed; The target vehicle speed is determined from the at least one candidate speed based on the third capture quality corresponding to each candidate speed; The first capture time for the target object is planned based on the target vehicle speed.
7. The snapshot method according to any one of claims 1 to 6, characterized in that, After acquiring at least one tracked object, the method further includes: A first capture request is generated when a preset condition is met; wherein, the first capture request is used to trigger the second camera to capture; the preset condition is that any pixel position of the tracked object in the image captured by the first camera intersects with a preset detection box; The second camera is controlled to capture images according to the first capture request.
8. The snapshot method as described in claim 7, characterized in that, The method further includes: If the first capture request and the second capture request exist simultaneously, it is determined whether the tracking object corresponding to the first capture request is the capture object; wherein, the second capture request is used to trigger the second camera to capture at the target capture time; If the tracking object corresponding to the first capture request is the capture object, then the first capture request is deleted.
9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the snapshot method as described in any one of claims 1 to 8.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the snapshot method as described in any one of claims 1 to 8.