A control method and device of an image acquisition device, an image acquisition device, and a storage medium

Abstract

Embodiments of the present application provide a kind of image acquisition equipment control method, device, equipment and storage medium, involve computer vision technical field.The specific implementation scheme is: the detection frame of each specified object contained in video frame is detected;For each detection frame, if there is stationary specified object matching the reid feature of the detection frame, then determine that the specified object in the detection frame and the stationary specified object are the same object;If there is motion specified object matching the reid feature and pixel position of the detection frame, then determine that the specified object in the detection frame and the motion specified object are the same object;The detection frame of the current specified object to be followed is determined from the video frame;According to the position of the detection frame of the current specified object to be followed, the pose of the image acquisition equipment is controlled, so that the current specified object to be followed is located in the video frame collected in preset position.Visibly, by the present scheme, image acquisition equipment can be controlled to accurately follow target rotation.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a control method, apparatus, device, and storage medium for an image acquisition device. Background Technology

[0002] In scenarios such as intelligent transportation, cameras are used to capture images of vehicles, pedestrians, and other objects on various roads. For target objects that meet the requirements within the captured images, the camera rotates to follow the movement of the target object during image acquisition, continuously capturing multiple frames containing that target object for subsequent analysis.

[0003] Because the camera rotates, the pixel positions of the same object in two consecutive frames captured by the camera will be significantly different. As a result, a stationary object may be identified as a moving object, and a stationary object in a new image frame may be mistaken for the target object, causing the camera to fail to follow the true target object.

[0004] Therefore, how to control the camera rotation to accurately follow the target object has become a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The purpose of this application is to provide a control method, apparatus, device, and storage medium for an image acquisition device, so as to control the image acquisition device to accurately follow the rotation of a target. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of this application provide a control method for an image acquisition device, the method comprising:

[0007] The detection bounding boxes of each specified object contained in the first video frame currently captured by the image acquisition device are used as the first detection bounding boxes;

[0008] For each first detection box, if there is a stationary specified object among the specified objects detected based on historical video frames that matches the re-identification features of the first detection box, then it is determined that the specified object in the first detection box and the matching stationary specified object are the same object.

[0009] If among the detected specified objects, there is a motion specified object that matches both the re-identification feature and pixel position of the first detection box, then it is determined that the specified object in the first detection box and the matching motion specified object are the same object;

[0010] Determine the first detection box of the specified object to be followed from the first video frame;

[0011] Based on the position of the first detection box of the specified object to be followed, the pose of the image acquisition device is controlled so that the specified object to be followed is located at a preset position in the acquired video frame.

[0012] Secondly, embodiments of this application provide a control device for an image acquisition device, the device comprising:

[0013] The detection module is used to detect the bounding boxes of each specified object contained in the first video frame currently captured by the image acquisition device, and use them as the first detection boxes;

[0014] The first determining module is used to determine that, for each first detection box, if there is a stationary specified object in the specified object detected based on historical video frames that matches the re-identification features of the first detection box, the specified object in the first detection box and the matching stationary specified object are the same object.

[0015] The second determining module is used to determine that if there is a motion-specific object in the detected specified object that matches both the re-identification features and pixel position of the first detection box, the specified object in the first detection box and the matching motion-specific object are the same object.

[0016] The third determining module is used to determine the first detection box of the specified object to be followed from the first video frame;

[0017] The control module is used to control the pose of the image acquisition device according to the position of the first detection box of the specified object to be followed, so that the specified object to be followed is located at a preset position in the acquired video frame.

[0018] Thirdly, embodiments of this application provide an electronic device, including:

[0019] Memory, used to store computer programs;

[0020] The processor, when executing a program stored in memory, implements the control method of the image acquisition device described in the first aspect above.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the control method for the image acquisition device described in the first aspect.

[0022] Fifthly, embodiments of this application provide a computer program product containing executable instructions that, when executed on a computer, cause the computer to perform the control method for the image acquisition device described in the first aspect.

[0023] Beneficial effects of the embodiments in this application:

[0024] The solution provided in this application addresses the issue that the pixel positions of stationary specified objects change during the rotation of the image acquisition device. Therefore, pixel position matching for stationary specified objects can be inaccurate. Instead, for each stationary specified object detected based on historical video frames, only ReID feature matching is used to perform ReID feature matching between each stationary specified object and the first detection box. The specified object in the first detection box that matches the ReID feature of the stationary specified object is considered the same object. However, for moving specified objects, since the camera rotates with the movement of the target, there is a relative speed between the moving specified object and the camera rotation. This allows pixel position matching to still be used to match each moving specified object and each first detection box. Therefore, by combining pixel position matching and ReID feature matching for moving specified objects, the first detection box matching the moving specified object can be accurately determined.

[0025] By employing different matching methods for stationary and moving specified objects, the identity of the specified object in each first detection frame can be accurately determined. This allows for the accurate identification of the first detection frame of the specified object to be followed from the first video frame. Based on the determined position of the first detection frame of the specified object to be followed, the pose of the image acquisition device is controlled, positioning the specified object at a preset position within the acquired video frame. This enables the image acquisition device to accurately follow the movement of the specified object during rotation.

[0026] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0028] Figure 1 A schematic diagram of a follow-up logic provided for an embodiment of this application;

[0029] Figure 2 A flowchart illustrating a control method for an image acquisition device provided in this application embodiment;

[0030] Figure 3 Another flowchart illustrating the control method for the image acquisition device provided in this application embodiment;

[0031] Figure 4 Another flowchart illustrating the control method for an image acquisition device provided in this application embodiment;

[0032] Figure 5 A schematic diagram of a covered scene for the control method of the image acquisition device provided in the embodiments of this application;

[0033] Figure 6A An image captured at the start of gimbal rotation, provided in an embodiment of this application;

[0034] Figure 6B An image captured during the rotation of a gimbal is provided in an embodiment of this application;

[0035] Figure 6C An image captured when the gimbal rotation ends, as provided in an embodiment of this application;

[0036] Figure 7 A schematic diagram illustrating a specific example of a control method for an image acquisition device provided in an embodiment of this application;

[0037] Figure 8 A schematic diagram of a data mining process provided in an embodiment of this application;

[0038] Figure 9 A schematic diagram of a ReID model provided in an embodiment of this application;

[0039] Figure 10 This is a schematic diagram of the structure of a control device for an image acquisition device provided in an embodiment of this application;

[0040] Figure 11 A block diagram of an electronic device for implementing the control method of the image acquisition device provided in the embodiments of this application. Detailed Implementation

[0041] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0042] The control method for an image acquisition device provided in this application can be applied to various electronic devices, such as servers and other devices with data processing capabilities. In one scenario, the electronic device executing the control method for the image acquisition device provided in this application can be the image acquisition device itself, that is, the control method for the image acquisition device is executed in the processor of the image acquisition device. It is understood that the control method for the image acquisition device provided in this application can be implemented by software, hardware, or a combination of software and hardware.

[0043] The control method for the image acquisition device provided in this application is geared towards a single-target following scenario. If a target (i.e., the specified object to be followed hereinafter) appears in the video frame acquired by the image acquisition device, then the target is continuously followed, that is, the lens of the image acquisition device is controlled to rotate as the target moves. Figure 1 As shown, when a target appears in the captured video frame, the target will be selected to follow. Then, as the target moves, the gimbal will be controlled to rotate according to the upward angle of the target (the angle between the target's velocity direction and the horizontal plane). When following ends or the target is lost, the target will be selected again.

[0044] like Figure 2 As shown, the control method for the image acquisition device provided in this application embodiment includes steps S201-S205:

[0045] S201, The detection boxes of each specified object contained in the first video frame currently acquired by the image acquisition device are used as the first detection boxes;

[0046] In this embodiment, the image acquisition device can rotate to follow the movement of a single target, continuously acquiring multiple video frames of that single target. This application embodiment does not limit the specific type of image acquisition device; for example, the image acquisition device can be a PTZ camera, a bullet camera, an IPC (IP Camera, network camera), etc. For example, in an intelligent traffic scenario, the image acquisition device can be a camera installed on a traffic light pole at an intersection. This camera acquires video frames of the current scene and uploads the acquired video frames to an electronic device that executes the control method for the image acquisition device provided in this application embodiment. The electronic device executing the control method for the image acquisition device provided in this application embodiment detects designated objects in the first video frame currently acquired by the camera, obtaining detection frames containing each designated object in the first video frame.

[0047] For example, the specified object can be set according to the application scenario. For instance, in a street scenario, the specified object can be set as a vehicle, and in a home scenario, the specified object can be set as a person.

[0048] For example, if the specified object is a person, a pre-trained object detection model for detecting people can be used to detect people in the first video frame, obtaining the detection boxes containing each person in the first video frame. If the specified object is a vehicle, a pre-trained object detection model for detecting vehicles can be used to detect vehicles in the first video frame, obtaining the detection boxes containing each vehicle in the first video frame. For example, the object detection model can be YOLOv5 (You Only Look Once version 5, a real-time object detection model) or SSD (Single Shot MultiBox Detector), etc. The specific type of object detection model is not limited in this application embodiment.

[0049] For example, an object detection model for detecting a specified object can be trained as follows: acquire sample images containing the specified object, with bounding boxes of each specified object labeled in the sample images; input the sample images into an initial object detection model to obtain the predicted bounding boxes output by the model, the confidence that the object exists within each predicted bounding box, and the confidence that the object belongs to the specified object; calculate the model loss value based on the difference between the model output and the labeled data; adjust the model parameters by backpropagation to minimize the model loss value until the model converges, thus obtaining an object detection model for detecting the specified object.

[0050] To ensure clarity of the solution layout, the following embodiments will introduce other implementation methods of the detection boxes of each specified object contained in the first video frame currently acquired by the detection image acquisition device, which will not be elaborated here.

[0051] S202, for each first detection box, if there is a stationary specified object that matches the reid feature of the first detection box among the specified objects detected based on historical video frames, then it is determined that the specified object in the first detection box and the matching stationary specified object are the same object.

[0052] In this embodiment, for each first detection box, the ReID (Re-identification) features of that first detection box can be extracted. For example, the image region containing each first detection box can be cropped from the first video frame as the image of each first detection box. Then, the cropped images of each first detection box are input into the ReID model to obtain the ReID features of each first detection box output by the model. The ReID model includes a feature extraction layer and a feature aggregation layer. The feature extraction layer enhances feature representation through residual connections and multi-scale feature fusion, enabling the extraction of multi-scale feature maps from the input image. The feature aggregation layer weighted and fused the multi-scale feature maps to generate a final feature vector, which serves as the ReID feature of the input image.

[0053] For example, after obtaining the REID features of each first detection box, for each first detection box, the vector similarity between the REID features of the first detection box and the REID features of each static specified object is calculated. Static specified objects with vector similarity greater than a preset threshold are identified as static specified objects that match the REID features of the first detection box. At this point, it is determined that the specified object contained in the first detection box and the matching static specified object are the same object, thus identifying the specified object contained in the first detection box. For example, the preset threshold could be 80% or 90%, etc.

[0054] For example, historical video frames can be a preset number of video frames preceding the currently acquired first video frame; this preset number can be 10, 20, etc. Based on historical video frames, it is possible to detect stationary specified objects and moving specified objects contained in the historical video frames.

[0055] For example, when the image acquisition device initially operates, the lens does not rotate because the target to be followed has not yet been determined. Therefore, based on adjacent frames in the acquired video frame sequence, stationary and moving specified objects can be detected within the video frames. For instance, moving specified objects can be identified by comparing the differences between adjacent frames using inter-frame difference analysis. That is, specified objects with pixel value differences close to 0 in adjacent frames are identified as stationary specified objects, while specified objects with larger pixel value differences in adjacent frames are identified as moving specified objects. Alternatively, multiple consecutive frames (e.g., 10 or 20 frames) in the video frame sequence can be acquired as a time queue, and the position of the center point of each detection box in the head and tail of the time queue can be used to determine whether movement has occurred. For example, if the pixel coordinate of the center point of detection box A in the head queue is coordinate A, and the center point of the detection box closest to coordinate A in the tail queue is coordinate B, if the Euclidean distance between coordinate A and coordinate B exceeds a predetermined threshold (e.g., 60 pixels), then the specified object in detection box A can be determined to be a moving specified object.

[0056] After identifying the initial stationary and moving specified objects, object information for both objects can be constructed and stored. For example, the object information for any specified object may include: identification, whether it is a stationary specified object, REID features, whether it is the currently following specified object, and motion path features. The REID features are obtained by extracting REID features from the specified object using a REID model. The REID features of any currently stored specified object can be obtained by extracting REID features from the current first video frame, or they can be a fusion of the REID features from the current first video frame and historical video frames. For example, this fusion feature can be a weighted sum of the first REID feature of the specified object in historical video frames and the second REID feature of the specified object in the current first video frame. The weights of the first and second REID features can be set by relevant technical personnel based on experience. In one implementation, the weight of the second REID feature is greater than the weight of the first REID feature to obtain REID features that incorporate more up-to-date information. Furthermore, among multiple historical video frames, the weight of the first REID feature of the specified object increases in the later the historical video frame was captured. The motion path feature includes the pixel position of the specified object in the current video frame and the velocity calculated based on the historical motion path of the specified object. The pixel position can be represented by the center point coordinates, width, and height.

[0057] Additionally, based on preset following rules, it can be determined whether each initial specified object is the current specified object to be followed. For example, the specified object occupying the largest area of ​​the video frame among all specified objects can be determined as the current specified object to be followed, or the specified object occupying the largest area of ​​the video frame among all moving specified objects can be determined as the current specified object to be followed; both are reasonable. After the current specified object to be followed is determined, the parameter value of "whether it is the current specified object to be followed" in the object information of that specified object can be updated to "yes". Furthermore, the following object is only updated when the replacement condition is met. For the sake of clarity in the scheme layout, the replacement condition will be described in the following text and will not be repeated here.

[0058] After obtaining the initial static and moving specified objects, each subsequent video frame can be captured. The REID features of the static specified objects in the currently stored object information can be matched with the REID features of each detection box in that video frame to determine whether each detection box contains the detected static specified object. Furthermore, after capturing each video frame, the object information of each specified object in the currently stored data can be updated based on the specified objects contained in that video frame. For example, if the object information for moving specified objects A and B is currently stored, but only moving specified object B exists in the first captured video frame, a disappearance count for moving specified object A can be performed. If the disappearance count reaches a preset disappearance duration threshold (e.g., 10s or 20s), the object information for moving specified object A can be deleted, and the object information for moving specified object B can be updated based on the latest information in the first video frame. For example, if the pixel position of moving specified object B changes in the video frame, the REID features and motion path features of moving specified object B can be updated.

[0059] Understandably, if the specified object is a vehicle, the stationary duration of a stationary vehicle is often longer than the tracking duration of the target being followed by the image acquisition device. For example, if the image acquisition device captures a stationary vehicle A in pose A, and when the image acquisition device finishes following vehicle B and returns to pose A, the stationary vehicle A still exists. Therefore, the object information for a stationary vehicle can be stored for a longer period. That is, for stationary vehicles detected in historical video frames, a longer time window can be set. Within this time window, even if the first video frame does not contain the stationary vehicle, its object information is still retained. This allows the image acquisition device to directly use the REID features in the stored object information for identification when it returns to a pose where it can capture the stationary vehicle within this time window, without needing to re-extract REID features from the stationary vehicle, thus improving matching efficiency. For example, this time window could be set to 5 minutes or 10 minutes, etc. In practical applications, the time window can be adjusted according to the product parameters of the actual product (i.e., the image acquisition device). For example, the time window can be set in association with the product's field of view and rotation speed. The rotation speed is negatively correlated with the size of the time window (i.e., the larger the rotation speed, the smaller the time window), while the field of view is positively correlated with the time window (i.e., the larger the field of view, the larger the time window).

[0060] S203, if among the detected specified objects, there is a motion specified object that matches both the reid feature and pixel position of the first detection box, then it is determined that the specified object in the first detection box and the matching motion specified object are the same object;

[0061] It is understandable that steps S202 and S203 are processed in parallel. That is, after obtaining the initial static and moving specified objects, upon acquiring each video frame, it is possible to determine whether there exists a moving specified object that matches the REID feature and pixel position of each detection box in that video frame, based on the object information of each moving specified object contained in the currently stored object information. This determines whether each detection box in that video frame contains the detected moving specified object. Furthermore, after acquiring each video frame, the object information of each specified object currently stored can be updated according to the specified objects contained in that video frame.

[0062] If the intersection-union ratio (IoU) between the predicted pixel position of any motion-specified object and the pixel position of the first detection box reaches a preset threshold, then the pixel position of the motion-specified object matches the pixel position of the first detection box. The predicted pixel position of the motion-specified object can be predicted based on the motion path features of the motion-specified object. For example, the preset threshold can be 80% or 90%.

[0063] In one implementation, step S203 above may include steps A1-A2:

[0064] A1: For each detected motion-specified object, obtain the predicted pixel position of that motion-specified object in the first video frame;

[0065] For example, for each detected moving object, the predicted pixel position of the moving object in the first video frame can be predicted using the motion path features (pixel position and velocity) of the moving object. For instance, the velocity of the moving object is multiplied by the frame interval to obtain the moving distance. Based on the velocity direction of the moving object, the pixel position of the moving object is moved by the moving distance in the velocity direction, and the resulting pixel position is the predicted pixel position.

[0066] A2, if the difference between the predicted pixel position corresponding to the motion specified object and the pixel position of the first detection box is less than a preset threshold, and the REID feature of the motion specified object matches the REID feature of the detection box, then it is determined that the specified object in the first detection box and the motion specified object are the same object.

[0067] For example, the intersection-over-union (IoU) ratio between the predicted pixel position of the specified moving object and the pixel position of the first detection box can be calculated. The difference between the predicted pixel position and the pixel position of the first detection box can be determined based on the IoU ratio. For instance, the difference between 1 and the IoU ratio can be used as this difference. In this case, the preset threshold can be 10% or 20%, etc. Alternatively, the pixel distance between the center points of the predicted pixel position and the pixel position of the first detection box can be used as the difference between them. In this case, the preset threshold can be set according to the width of the video frame, such as 10% or 5% of the frame width, which are both reasonable.

[0068] For example, the vector similarity between the REID features of the specified moving object and the REID features of the detection box can be calculated. If the vector similarity is greater than a specified threshold, then the REID features of the specified moving object are determined to match the REID features of the detection box. For example, the specified threshold could be 80% or 90%, etc.

[0069] In other words, in this implementation, the specified object in the first detection box and the specified motion object are determined to be the same object only when the difference between the predicted pixel position and the pixel position of the first detection box is less than a preset threshold, and the REID feature matches the REID feature of the detection box.

[0070] In another approach, for each first detection box, a first similarity can be calculated between the reid features of each detected motion-designated object and the reid features of the first detection box. Then, based on the motion path features of the motion-designated object, the predicted pixel position of the motion-designated object in the first video frame is predicted, and a second similarity is calculated between the pixel position of the first detection box and the predicted pixel position. Combining the first similarity and the second similarity, the motion-designated object matching the first detection box is determined.

[0071] For example, the first similarity between REID features can be obtained by calculating the vector similarity between REID features. The predicted pixel position of the motion-specified object is the position of the predicted bounding box, which can be represented by the center point, width, and height. The second similarity between the pixel position of the first detection box and the predicted pixel position of the motion-specified object can be obtained by calculating the intersection-union ratio (IUU) between the pixel position of the first detection box and the predicted pixel position of the motion-specified object.

[0072] For example, the method for determining the motion-designated object matching the first detection box by combining the first similarity and the second similarity can be: performing a weighted sum of the first similarity and the second similarity, and determining the motion-designated object whose weighted sum of the first similarity and the second similarity is greater than a preset similarity threshold as the motion-designated object matching the first detection box; wherein, the weights of the first similarity and the second similarity can be set by relevant technical personnel based on experience. For example, the preset similarity threshold can be 80% or 90%, etc.

[0073] In another implementation, motion-designated objects whose pixel positions differ from those of the first detection box by a preset threshold are first identified from among the motion-designated objects. After obtaining the motion-designated objects whose pixel positions differ from those of the first detection box by a preset threshold, the REID features of the obtained motion-designated objects are matched with the REID features of the detection box, and the motion-designated objects that match the REID features of the detection box are identified as the same objects as the designated objects in the first detection box.

[0074] Understandably, this method first filters out motion-specified objects from all motion-specified objects that match the pixel positions of the first detection box. Then, it further filters out motion-specified objects that match the REID features of the first detection box, resulting in motion-specified objects that match both the REID features and pixel positions of the first detection box. This way, for motion-specified objects whose pixel positions differ from the first detection box by a preset threshold, there's no need to further match the REID features of the motion-specified object with those of the detection box, saving computational resources. Furthermore, since IOU (Intersection over Union) matching has a relatively small computational cost, using IOU matching for rapid filtering before matching the IOU-matched motion-specified objects with the REID features of the first detection box improves matching speed.

[0075] S204, Determine the first detection box of the specified object to be followed from the first video frame;

[0076] It is understandable that, through the aforementioned steps S202 and S203, the identities of each detected specified object and the specified objects contained in each first detection frame can be associated, thereby determining the identities of some of the specified objects in the first detection frames. Therefore, if the first video frame contains the latest following specified object detected based on historical video frames, the first detection frame of the latest following specified object can be determined as the first detection frame of the specified object to be followed.

[0077] In one implementation, before performing step S204 to determine the first detection box of the specified object to be followed from the first video frame, the method further includes: if there is no latest specified object to be followed in the first video frame, the loss duration of the latest specified object to be followed is accumulated.

[0078] It is understandable that the latest following specified object is the following object detected based on the historical video frames of the first video frame. That is, among the currently stored object information of each specified object, the specified object whose parameter value for "whether it is the currently following specified object" is "yes" is the latest following specified object. If the latest following specified object does not exist in the first video frame, it means that the latest following specified object has been lost. In this case, the loss duration of the latest following specified object can be accumulated.

[0079] Accordingly, in this implementation, the step S204 above, which determines the first detection box of the specified object to be followed from the first video frame, includes:

[0080] If the accumulated loss time reaches the first preset time threshold, select the first detection box that contains the specified moving object and has the largest area as the first detection box of the specified object to be followed.

[0081] Accordingly, in this implementation, the above method may further include:

[0082] If the accumulated loss time reaches the second preset time threshold, the first detection box with the largest area is selected as the first detection box of the specified object to be followed; wherein, the second preset time threshold is greater than the first preset time threshold.

[0083] In other words, when the accumulated lost time reaches the first preset time threshold, and a moving specified object exists in the current first video frame, the replacement condition is met, and the latest specified object to be followed needs to be replaced. That is, the first detection box of the moving specified object with the largest area in the current first video frame is selected as the first detection box of the specified object to be followed.

[0084] If the accumulated lost time reaches a first preset time threshold and no moving specified object exists in the current first video frame, the lost time of the latest followed specified object continues to be accumulated. After the lost time reaches the first preset time threshold, moving specified objects in the acquired first video frames are continuously detected. When a moving specified object appears in the first video frame, the detection box of the moving specified object with the largest area can be selected as the first detection box of the specified object to be followed. If no moving specified object appears when the accumulated lost time reaches a second preset time threshold, another replacement condition is met. At this time, the first detection box with the largest area in the current first video frame is selected as the first detection box of the specified object to be followed.

[0085] For example, the first preset duration threshold can be 1 second or 2 seconds, etc. The second preset duration threshold can be 3 seconds or 5 seconds, etc. The second preset duration threshold is greater than the first preset duration threshold. For example, in one implementation, the first preset duration threshold is 1 second and the second preset duration threshold is 3 seconds.

[0086] Understandably, by setting a first duration threshold greater than a second preset duration threshold, once the first preset duration threshold is reached, if a moving specified object exists, it can be followed first. If no moving specified object exists, it can be switched to a stationary specified object after waiting for a longer period of time. This effectively prevents the latest specified object from being lost (occluded or blurred) during the follow and switches to a stationary specified object in a short time, which better meets the needs of the follow scenario that focuses more on the moving specified object.

[0087] S205, based on the position of the first detection box of the specified object to be followed, control the pose of the image acquisition device so that the specified object to be followed is located at a preset position in the acquired video frame.

[0088] In this embodiment, the preset position can be the center position in the video frame. That is, the pose of the image acquisition device is adjusted according to the position of the first detection box of the specified object to be followed, so that the specified object to be followed can be located in the center of the screen. Then, the image acquisition device can rotate to follow the position of the specified object to be followed.

[0089] For example, if the image acquisition device includes a lens and a pan-tilt unit (PTZ) supporting the lens, the horizontal and vertical offsets between the center points can be calculated based on the coordinates of the center point of the first video frame acquired by the lens and the coordinates of the center point of the first detection frame of the specified object to be followed. Then, the PTZ rotation is controlled based on the magnitude and direction of these two offsets. For instance, in practical applications, various methods such as PID (Proportional-Integral-Derivative) control and Kalman filtering can be used to control the PTZ rotation; this embodiment is not limited to these methods.

[0090] In another embodiment of this application, a pre-trained specified object detection model can be used to detect the first video frame, obtaining detection boxes for each specified object contained in the first video frame. The specified object detection model is trained using first specified sample images; for example... Figure 3 As shown, the method for obtaining the first specified sample image includes steps S301-S302:

[0091] S301, based on the scene image associated with the specified scene where the image acquisition device is located, obtain query reference data; wherein, the query reference data includes: descriptive text used to describe a specified object in the specified scene, and / or, a template image containing the specified object in the specified scene;

[0092] S302, determine the image from the sample image that matches the features of the query reference data, and obtain the first specified sample image.

[0093] In this embodiment, a scene image associated with a specified scene where the image acquisition device is located can be acquired first. For example, if the specified scene where the image acquisition device is located is traffic intersection A, then the scene image of traffic intersection A can be acquired as the scene image associated with the specified scene where the image acquisition device is located.

[0094] For example, after obtaining the scene image associated with a specified scene, a graph-to-text model can be used to generate descriptive text for the scene image associated with that specified scene. Then, descriptive text related to the specified object can be extracted from the generated descriptive text as query reference data. For example, if the specified object is a vehicle, and the scene image associated with the specified scene contains a bus and a motorcycle, the generated descriptive text will contain descriptions related to "bus" and "motorcycle." Using keyword matching, descriptive text related to "vehicle" can be extracted, thus obtaining "bus" and "motorcycle" as query reference data.

[0095] For example, after obtaining the scene images associated with a specified scene, an image containing a specified object can be selected from the scene images associated with each specified scene as a template image. For instance, if the specified object is a vehicle, and the obtained scene images associated with the specified scene include images 1-10, images 1 and 10 do not contain vehicles, while images 2-9 contain vehicles, then images 2-9 can be used as template images.

[0096] It is understandable that the more first specified sample images a model has when training a specified object detection model, the better the training effect. Therefore, the sample images can be expanded to increase the number and richness of the sample images, thereby increasing the number and richness of the first specified sample images determined from the sample images.

[0097] For example, video frames captured by image acquisition devices in various scenes can be directly obtained by establishing a data feedback mechanism, and a synthetic image of the directly acquired video frames can be generated using data synthesis. For instance, the acquired video frames can be input into a generative adversarial network to generate new sample images, which are then used as synthetic images. The directly acquired video frames and the synthetic images can be used as sample images to enrich the sources of sample images.

[0098] If the aforementioned query reference data includes descriptive text, the method for determining images from sample images that match the features of the query reference data includes: for each sample image, generating descriptive text for that sample image using a graph-to-text model, and encoding the generated descriptive text using a text encoder to obtain a text vector for each sample image; text encoding the descriptive text in the query reference data to obtain a text vector for the query reference data; calculating a first similarity between the text vector of each sample image and the text vector of the query reference data, and determining sample images with a first similarity greater than a preset threshold as images that match the features of the query reference data.

[0099] If the aforementioned query reference data includes template images, the method for determining images from sample images that match the features of the query reference data includes: for each sample image, extracting the image vector of the sample image to obtain the image vector of each sample image; extracting the image vector of the template image in the query reference data in the same way as extracting the image vector of the sample image to obtain the image vector of the query reference data; calculating the second similarity between the image vector of each sample image and the image vector of the query reference data, and determining the sample images with the second similarity greater than a preset threshold as images that match the features of the query reference data.

[0100] If the aforementioned query reference data includes descriptive text and template images, after calculating the first similarity between the text vector of each sample image and the text vector of the query reference data, and the second similarity between the image vector of the sample image and the image vector of the query reference data, the first and second similarities can be weighted and summed to obtain the final similarity between the sample image and the query reference data. Sample images with a final similarity greater than a preset threshold are identified as images that match the features of the query reference data. For example, the preset threshold can be 80% or 90%, etc. The weights of the first and second similarities can be set by relevant technical personnel based on experience.

[0101] For example, in one implementation, a large-scale multimodal database can be established using data backflow and data synthesis. This database stores sample images, and each sample image is associated with multimodal features. In practical applications, a multimodal model can be used to extract the image vector of each sample image as its image feature, and a graph-to-text model can be used to generate descriptive text for the sample image. This descriptive text is then encoded into a text vector using a text encoder to obtain the text feature of the sample image. The image feature and the text feature are then stored as multimodal features associated with the sample image.

[0102] Accordingly, in this implementation, text vectors and / or image vectors of the query reference data can be extracted as described above and used as features of the query reference data. Then, the features of the query reference data are directly used as the query to retrieve data from the multimodal database. Sample images with a similarity greater than a preset threshold are selected as images that match the features of the query reference data. By searching in this pre-built multimodal database, images that match the features of the query reference data can be retrieved quickly.

[0103] Understandably, since the reference data is obtained based on scene images associated with a specified scene where the image acquisition device is located, retrieving images from the sample images that match the features of the reference data allows for the selection of sample images related to the specified scene. Training the specified object detection model using these selected sample images improves the model's accuracy in detecting specified objects in images related to that specified scene. Therefore, the trained specified object detection model can accurately detect the bounding boxes of each specified object contained in the first video frame currently captured by the image acquisition device.

[0104] In one implementation, after identifying an image from the sample images that matches the features of the query reference data, this identified image can be directly used as the first specified sample image. In another implementation, to further improve the training performance of the specified object detection model, the identified image can be further processed to improve the quality of the first specified sample image.

[0105] For example, the first specified sample image can be obtained in the following four ways:

[0106] Method 1: Determine the image from the sample image that matches the features of the query reference data to obtain the second specified sample image; perform image enhancement processing on the obtained second specified sample image, and combine the second specified sample image and the image enhancement processed image to obtain the first specified sample image.

[0107] In this method, an image that matches the features of the query reference data is determined from the sample images as the second specified sample image, the second specified sample image is subjected to image enhancement processing, and the second specified sample image and the image after image enhancement processing are used as the first specified sample image.

[0108] For example, image enhancement processing includes Gaussian blurring and / or sharpening, etc. Using the second specified sample image and the image enhanced as the first specified sample image can improve the richness of the training data. Furthermore, by applying Gaussian blurring to the second specified sample image, image details can be reduced. Therefore, training a specified object detection model using the Gaussian-blurred image allows the model to learn more robust features (such as shape features rather than texture features), making the trained model applicable to real-world scenes with large lighting variations or slight blur. Similarly, by sharpening the second specified sample image, edge contrast can be enhanced. Therefore, training a specified object detection model using the sharpened image can improve the model's sensitivity to details. Combining Gaussian blurring and sharpening can simulate different observation conditions (such as blurred + sharp). Training a specified object detection model using images with both Gaussian blurring and sharpening can improve the model's generalization ability, enabling it to adapt to more complex environments.

[0109] Method 2: Determine images from the sample images that match the features of the query reference data to obtain second specified sample images; score the second specified sample images according to preset scoring dimensions to obtain the quality score of each second specified sample image; select a preset number of second specified sample images with the highest quality scores as first specified sample images; the preset scoring dimensions include the completeness, size and / or orientation of the specified objects contained in the scene image.

[0110] In this method, after obtaining each second specified sample image, each second specified sample image can be scored according to the completeness, size, and / or orientation of the specified object contained in the image. For example, the completeness and size of the specified object are positively correlated with the score, that is, the higher the completeness of the specified object, the higher the score; the larger the size of the specified object, the higher the score; the orientation of the specified object is related to the score as follows: frontal > lateral > back.

[0111] If there is only one preset scoring dimension, the score obtained from that dimension can be directly used as the quality score. If there are multiple preset scoring dimensions, for each second specified sample image, after obtaining the score obtained from scoring each preset scoring dimension, the scores of each preset scoring dimension are weighted and summed, and the weighted sum is used as the quality score. In practical applications, relevant technical personnel can set the weights of each preset scoring dimension based on experience.

[0112] After obtaining the quality scores of each second specified sample image, a preset number of second specified sample images with the highest quality scores can be selected as the first specified sample images, in descending order of quality scores. For example, the preset number can be 1000 or 10000, etc.

[0113] Understandably, using the completeness of a specified object as a preset scoring dimension can filter out images with high completeness. Training the model with these filtered images allows the model to learn more comprehensive features of the specified object, thereby reducing false detections caused by occlusion. Using the size of a specified object as a preset scoring dimension can filter out images containing larger specified objects. Training the model with these filtered images allows the model to be applicable to the detection of large-scale specified objects, which is suitable for practical following scenarios where it is not necessary to detect very small specified objects. Using the orientation of a specified object as a preset scoring dimension can filter out images containing more positively oriented specified objects. Training the model with these filtered images can improve the model's ability to recognize specified objects from a positive perspective. Combining completeness, size, and orientation can filter out clear sample images without severe occlusion. Training the specified object detection model with these selected sample images can achieve faster convergence speed and better detection results.

[0114] For example, the method of scoring the second specified sample image according to a preset scoring dimension can be achieved by using a pre-trained quality score model. The second specified sample image is input into the quality score model to obtain the quality score output by the model. This quality score model is trained based on specified training samples, each of which is labeled with its own quality score. The specified training sample image is input into the initial quality score model to obtain the predicted score output by the model. The model loss value is calculated based on the difference between the model's output and the labeled data. The model parameters are adjusted by backpropagation to minimize the model loss value until the model converges, resulting in the quality score model used for scoring. For example, the initial quality score model can be a CNN (Convolutional Neural Network) or a DNN (Deep Neural Network), etc. The specific type of the initial quality score model is not limited in this embodiment.

[0115] Method 3: Determine the image from the sample images that matches the features of the query reference data to obtain the second specified sample image; perform image enhancement processing on the obtained second specified sample image, and combine the second specified sample image and the image enhancement processed image to obtain the third specified sample image; score the third specified sample image according to the preset scoring dimensions to obtain the quality score of each third specified sample image; select the preset number of third specified sample images with the highest quality scores as the first specified sample images; the preset scoring dimensions include the completeness, size and / or orientation of the specified objects contained in the scene image.

[0116] Method 4: Determine images from the sample images that match the features of the query reference data to obtain second specified sample images; score the second specified sample images according to preset scoring dimensions to obtain the quality score of each second specified sample image; select a preset number of second specified sample images with the highest quality scores as fourth specified sample images; the preset scoring dimensions include the completeness, size and / or orientation of the specified objects contained in the scene image; perform image enhancement processing on the obtained fourth specified sample images, and combine the second specified sample images and the image enhancement processed images to obtain the first specified sample image.

[0117] Methods three and four described above combine image enhancement and scoring / selection to obtain the final first specified sample image. The descriptions of image enhancement and scoring / selection are similar to those in methods one and two, and will not be repeated here. By combining these two methods, the image quality of the final first specified sample image can be further improved, thereby increasing the accuracy of the specified object detection model trained using the first specified sample image.

[0118] In one implementation, an initial specified object detection model is obtained by performing at least one of the following model adjustment processes on the first specified YOLOV5 model: (1) replacing the convolutional layers in the backbone network with depth separation convolutional layers; (2) reducing the number of parameters after the FPN (Feature Pyramid Networks); (3) adding a 64x downsampling layer in the backbone network; (4) replacing the C3 layer in the backbone network with the SEC3 layer; and (5) increasing the convolutional kernel of the first convolutional layer.

[0119] Understandably, by replacing the convolutional layers in the backbone network of the first specified YOLOv5 model with depthwise separating convolutional layers, the number of parameters in the convolutional layers of the backbone network can be reduced from... Reduce to This significantly reduces the computational cost of the model, achieving lightweight design while maintaining detection accuracy. Here, k is the kernel size, c_in represents the number of input channels, and c_out represents the number of output channels.

[0120] For example, the number of channels after the First Processing Neural Network (FPN) of the first specified YOLOv5 model can be compressed to reduce the number of parameters after the FPN. For instance, the number of channels after the FPN can be compressed from 256 to 128, halving the number of parameters after the FPN. In practical applications, channel pruning can be used to remove channels in the FPN whose weights in all Batch Normalization (BN) layers are less than a set threshold, thereby compressing the number of channels after the FPN of the first specified YOLOv5 model and reducing the number of parameters in subsequent operations.

[0121] For example, convolutional or pooling layers can be added after the existing 32x downsampling layer in the first specified YOLOv5 model. For instance, a convolutional layer or max-pooling layer with a stride of 2 can be added after the existing 32x downsampling layer to achieve further downsampling, thus adding a 64x downsampling layer to the backbone network. When adding a 64x downsampling layer, it is necessary to ensure that the number of input and output channels of the newly added 64x downsampling layer matches that of the preceding and following layers to avoid feature dimension conflicts.

[0122] The SEC3 layer is an improved C3 module that incorporates a channel attention mechanism. The SEC3 layer includes a convolutional layer with an SE (Squeeze-and-Excitation) module and a C3 layer. By introducing the SE module into the standard convolution, the feature representation capability can be enhanced. By replacing the C3 layer in the backbone network of the first specified YOLOv5 model with the SEC3 layer, the model's ability to represent features at different scales can be improved.

[0123] For example, increasing the size of the convolutional kernel of the first convolutional layer could be done by... The convolution kernel is adjusted to By increasing the size of the convolutional kernel in the first convolutional layer, the model's ability to perceive a wider range of information and edge features in the image can be improved, thus increasing the model's receptive field. In practical applications, the range of increase in the convolutional kernel of the first convolutional layer can be adjusted by relevant technicians based on experience or model training results, and this application embodiment does not limit this.

[0124] It is understandable that by adopting the above methods (1) or (2), the number of parameters of the first specified YOLOv5 model can be reduced, making the adjusted model suitable for low-computing-power scenarios. By adopting any of the above methods (3)-(5), the model's ability to identify larger targets can be improved, making the adjusted model suitable for scenarios that require following relatively close specified objects. For example, if it is necessary to follow a relatively close vehicle, since the relatively close vehicle occupies a larger area of ​​the screen, the model needs to be able to identify large-scale targets.

[0125] In one implementation, the pre-trained specified object detection model is obtained by training a second specified YOLOv5 model using the model after model adjustment as the teacher model; the number of parameters of the second specified YOLOv5 model is less than the number of parameters of the model after model adjustment.

[0126] In this implementation, the number of parameters in the second specified YOLOv5 model is less than the number of parameters in the model after model adjustment. For example, the first specified YOLOv5 model can be VOLOv5x, with approximately 26.1M parameters. If the number of parameters in the model after adjustment is 20M, then the second specified YOLOv5 model can be VOLOv5m, with approximately 11.3M parameters; or, the second specified YOLOv5 model can be VOLOv5l, with approximately 16.5M parameters.

[0127] For example, the method of training a student model using a teacher model includes: first, training the teacher model using the first specified sample image; after obtaining a converged teacher model, training the student model using the teacher model, that is, inputting training samples into the student model and the trained teacher model respectively; adjusting the model parameters of the student model based on the difference between the outputs of the student model and the teacher model until the difference between the outputs of the student model and the teacher model is less than a preset threshold, thus obtaining a trained student model, which serves as the specified object detection model. For example, the training method of the teacher model can refer to the relevant description of the training method of the object detection model described above, and will not be repeated here.

[0128] It is understandable that by using the model after model adjustment as the teacher model to train the second specified VOLOV5 model, the second specified VOLOV5 model can learn the teacher model's ability to detect specified objects. Subsequently, using the trained second specified VOLOV5 model as the specified object detection model can further reduce the model's parameter size, enabling the specified object detection model to be deployed on low-computing-power devices and thus applied in low-computing-power scenarios.

[0129] In another implementation, the model obtained after the aforementioned model adjustment process aimed at reducing the number of parameters can be used as the student model, and the first specified YOLOv5 model can be used as the teacher model. The student model can then be trained using the teacher model to obtain the specified object detection model. The model adjustment process aimed at reducing the number of parameters includes: replacing the convolutional layers in the backbone network of the first specified YOLOv5 model with depthwise separating convolutional layers, and reducing the number of parameters in the first specified YOLOv5 model's FPN.

[0130] It is understandable that by using the first specified YOLOv5 model as the teacher model, and training the model obtained after model adjustment with the aim of reducing the number of parameters, the model obtained after model adjustment can learn the detection capability of the first specified YOLOv5 model for the specified object. Subsequently, the model obtained after adjustment of the trained model can be used as the specified object detection model, which can reduce the parameter size of the model and enable the specified object detection model to be deployed on low computing power devices, thus enabling its application in low computing power scenarios.

[0131] In another embodiment of this application, the ReID features of each detection box and the specified object are obtained by feature extraction using a ReID model; the ReID model is trained using a second specified sample image.

[0132] like Figure 4 As shown, the second specified sample image is obtained in the following ways:

[0133] S401, selects multiple sample video frame sequences based on high and low threshold strategies;

[0134] S402, based on the selected sample video frame sequence, obtain a second specified sample image; wherein, the similarity between image features of the same specified object contained in any selected sample video frame sequence is greater than a preset similarity threshold.

[0135] For example, multiple sample video frame sequences can be selected from a candidate sample database according to a high and low threshold strategy. The candidate sample database stores multiple candidate video frame sequences, which can be acquired by an image acquisition device located in a specified scene.

[0136] For example, when selecting sample video frame sequences from a candidate sample database, a high-low threshold strategy is used for high-threshold screening and low-threshold supplementation. Specifically, a high-confidence threshold (e.g., 0.8) can be set to recall candidate video frame sequences from each candidate video frame sequence where the similarity between image features containing the same specified object is higher than the high-confidence threshold. The recalled frame sequences are high-quality frame sequences. A low-confidence threshold (e.g., 0.3) can be set to recall candidate video frame sequences from each candidate video frame sequence where the similarity between image features containing the same specified object is higher than the high-confidence threshold. The recalled frame sequences are sequences that may be occluded or blurred. Combining the frame sequences recalled by the high-confidence threshold and the low-confidence threshold yields the selected sample video frame sequence, thus increasing data diversity.

[0137] For example, after obtaining the selected sample video frame sequence, the selected sample video frame sequence can be used as the second specified sample image. Alternatively, image enhancement processing can be performed on each video frame in each video frame sequence, and the selected sample video frame sequence and the sample video frame sequence obtained after image enhancement processing can be used as the second specified sample image to further improve the richness of the training data.

[0138] In one implementation, the reid model is obtained by connecting the SR module between the full-scale feature learning module and the output layer of the OSNet (Omni-Scale Network) model.

[0139] In this implementation, when introducing the SR (super-resolution) module into the OSNet model, the SR module is connected between the Omni-Scale Block (full-scale feature learning) module and the output layer. That is, the SR module is inserted after feature extraction is completed and before final classification.

[0140] Understandably, by connecting the SR module between the full-scale feature learning module and the output layer of the OSNet model, the adjusted OSNet model can first extract multi-scale features from the image through multiple Omni-Scale Blocks. These features are then fed into the SR module for super-resolution reconstruction to restore and enhance the image's detail information. Finally, these enhanced features are passed through global average pooling layers and fully connected layers for final classification or recognition. This design allows the OSNet model to retain its powerful feature discrimination capabilities while incorporating the image detail restoration function of SR technology, making it suitable for scenarios with low-quality or blurry input images.

[0141] In addition, to adapt to low computing power scenarios, after adjusting the OSNet model, the trained adjusted OSNet can be used as the teacher model to train a small-scale OSNet model with fewer parameters than the adjusted OSNet model, and the trained small-scale OSNet model can be used as the final reid model.

[0142] To better understand the control method of the image acquisition device provided in the embodiments of this application, a specific example is described below.

[0143] This example is geared towards following scenarios, such as... Figure 1 As shown, when a target appears in the captured video frame, the target will be selected to follow. Then, as the target moves, the gimbal will be rotated according to the upward angle of the target. When following ends or the target is lost, the target will be selected again.

[0144] If multiple bounding boxes are detected in the captured video frame, this example can automatically select the largest bounding box as the tracking target, and the camera will rotate as the target moves. The largest bounding box is selected because it is the most prominent target, which better meets the user's need to track prominent targets, resulting in a better user experience. After selecting the target, it needs to be continuously tracked to keep the same target on track. When tracking ends or the target is lost, a new target needs to be selected or a timer for target loss needs to be entered.

[0145] This example addresses a wide range of scenarios and has relatively high computational power requirements. For example... Figure 5 The image shown illustrates the coverage scenario for image acquisition in this example, including various device types, application scenarios, and imaging effects. See also... Figure 5This example is applicable to the following device types: integrated cameras (such as PTZ cameras, dome cameras, bullet cameras), mobile phones (HD and LHD), and other devices such as network camera devices (including indoor and outdoor devices), peephole cameras, robot vacuum cleaners, fisheye robots, and companion robots, etc. Application scenarios for this example can include enclosed scenes (such as indoor home scenes, underground garage scenes, office building scenes), semi-enclosed scenes (family yards, farms, factory workshops), and open scenes (such as farmland scenes, traffic arteries, construction sites). It is suitable for various imaging effects, including imaging modes of white light, black and white night vision, and full-color night vision; supplementary lighting intensity includes strong supplementary lighting and weak supplementary lighting; sensors include strong light sensitivity and weak light sensitivity; and imaging angles include suspended, level, and upward-looking.

[0146] The following example, using vehicle following in a traffic scenario, will illustrate this concept. This example of vehicle following has the following characteristics:

[0147] (1) Single target following, and priority is given to following moving targets.

[0148] (2) The camera follows the vehicle. The entire vehicle following process is more complex, requiring the design of the entire system and each module within the system to counteract the new problems brought about by the rotation of the gimbal.

[0149] (3) It is instantaneous and does not require long-term memory.

[0150] Currently, in practical business scenarios, the results often fall short of expectations, primarily because IOU matching fails when the gimbal rotates. When the target is at the edge of the frame, the tracking library reports a large rotation angle to drive the gimbal to rotate, thus centering the target. In this case, the change between the two frames becomes very large due to the gimbal rotation, causing IOU matching to fail. In some cases, target A in frame T-1 may even be matched to target B in frame T via IOU. When the camera needs to rotate a large angle, the gimbal rotation process is completed in multiple steps. However, during this process, the image is blurred, and almost no target is detected. Therefore, the targets currently fed into the tracking library often do not include targets generated during the rotation process. For example, from the start to the end of the gimbal rotation... Figures 6A-6C As shown. Figure 6A This is an image of an underground parking lot at the start of the gimbal rotation, with the vehicle positioned slightly to the left of the center of the image. Figure 6B The image was captured during the gimbal's rotation. It can be seen that the gimbal is rotating to the left to gradually center the vehicle in the image. Figure 6C This is the image when the gimbal finishes rotating; at this point, the vehicle is centered in the frame.

[0151] Currently, video frames captured during the rotation process are typically discarded to ensure the detection module's effectiveness. However, after discarding these frames, the video frames captured at the 'start of gimbal rotation' and 'end of gimbal rotation' in the above image show a significant difference. See also... Figure 6A and Figure 6C As can be seen, after removing the blurry frames captured during the intermediate rotation, the displacement between the two frames is very large, making it impossible for the vehicle in the image to follow through IOU matching.

[0152] To address the aforementioned issues, this example demonstrates a control method for an image acquisition device that primarily comprises a vehicle detection module and a vehicle following module. The vehicle following module utilizes a vehicle REID module, which serves as the feature matching component within the vehicle following module. An overall following diagram is shown below. Figure 7 As shown, the vehicle detection module detects vehicles in the acquired video frames. Specifically, it extracts features from the acquired video frames to obtain multi-scale feature maps. Then, it fuses the multi-scale feature maps and inputs them into the classification branch for detection, obtaining the detection boxes for each vehicle in the video frame. The detected detection boxes are then transmitted to the vehicle following module, which follows the vehicle according to a predetermined following logic.

[0153] like Figure 7 As shown, if four frames of images are acquired, the target selection for following can be performed in the first frame. After detecting dynamic vehicle D1 and static vehicles S1 and S2, D1 is selected to follow. The second frame is an image during the following process of D1. At this time, IOU matching and reid feature matching are used for D1, and reid feature matching is used for S1 and S2. The third frame is an image acquired when the following process of D1 is about to end. At this time, vehicles S1 and S2 disappear from the image. IOU matching and reid matching are used for D1, and the disappearance count of S1 and S2 is recorded. New vehicles N1, N2, and N3 that appear during the rotation are ignored. The fourth frame is an image acquired when the following of D1 ends. After waiting for a set time, the target for following is reselected. The disappearance count of S1 and S2 is recorded. For new vehicles N1, N2, and N3, the target is switched to the moving vehicle N3 after waiting for 1 second.

[0154] The following is a detailed design of each module:

[0155] (a) Vehicle detection module.

[0156] Compared to conventional vehicle inspection, the vehicle inspection module mainly needs to solve the following problems:

[0157] 1. It needs to cover multiple scenarios.

[0158] This example demonstrates a relatively complete data mining workflow designed to address this problem and cover different application scenarios. For example... Figure 8 As shown, the computing power, storage, and network resources of the data center provide underlying support for the AI ​​(Artificial Intelligence) capability matrix. Data sources are enriched by establishing a large-scale multimodal database for data feedback and data synthesis. Multimodal queries (corresponding to the query reference data mentioned above) are generated to enrich the training data retrieved from the multimodal database. The retrieved data is then sorted and filtered to obtain the final training data. Queries are primarily generated from two dimensions: text and images, to find training data (corresponding to the first specified sample image mentioned above). For example, the text 'vehicle' can be expanded to 'car', 'vehicle', 'bus', etc. For the image dimension, one or more 'template images' are selected from the relevant scenes as the query.

[0159] The quality score model is used to further mine high-quality samples that are relevant to the scene and output them to the model for training. For example, in practical applications, it is not necessary to detect some local or very small vehicles, which can be used as a dimension of quality.

[0160] 2. A smaller model is required.

[0161] (1) Replace the standard conv (convolution) layers in the backbone network of the YOLOv5 model with depthwise separable convolutions. The parameter computation is reduced from Reduced to .

[0162] (2) Based on actual observation, the model was pruned and optimized. Through tool analysis, the effective parameters were concentrated in the low-level convolutional layers. Therefore, for the YOLOV5 model, the parameters before the C3 backbone were retained, but the number of parameters after C3 was halved (i.e., the number of parameters after FPN was halved as mentioned above). Through experiments, it was found that the number of parameters was reduced by 50%, while the accuracy remained almost unchanged.

[0163] (3) The learning method of distillation is adopted. The Teacher model is given more training data through the data acquisition engine. The Teacher model constrains the Student model through the logit (logistic regression) layer, thereby improving the performance of the Student model.

[0164] 3. It needs to be robust to motion-blurred scenes (gimbal rotation, vehicle speed).

[0165] Various strategies, such as training data augmentation (corresponding to image enhancement processing mentioned above), are employed to enhance the richness of the training data.

[0166] 4. Depending on business needs, it may be necessary to detect vehicles that are relatively close (e.g., 3-5 meters away). In this case, the vehicles generally occupy a larger portion of the image.

[0167] (1) Improve the standard YOLOv5 model by adding a more abstract feature map with a downsampled size of 64 to detect large targets, in addition to performing regression on the original three downsampled feature maps of 32, 16, and 8.

[0168] (2) Adding an SE module to the standard convolution to improve the feature representation capability at different scales.

[0169] (3) Increase the size of the convolution kernel in the first conv layer to improve the receptive field.

[0170] (ii) Vehicle REID module.

[0171] Compared to vehicle detection modules, vehicle REID modules also face challenges such as diverse application scenarios and limited computing power.

[0172] (1) It needs to cover multiple scenarios.

[0173] The training data mining process is largely similar to that of vehicle detection, but the difference lies in that vehicle REID primarily mines the same ID (identity) from videos, rather than just from images. Image detection-based data mining methods often discard occluded or low-quality samples due to confidence limitations, but these samples can actually increase the robustness of the REID model. This example uses a high-low threshold strategy to recall some occluded and low-quality samples, significantly increasing the richness of the training data for the REID model.

[0174] (2) A smaller model is required.

[0175] We use a lightweight OSNet as the Student and a larger OSNet with more parameters as the Teacher. We also employ a distillation method to allow the Teacher to assist the Student in learning.

[0176] (2) It needs to be robust to motion-blurred scenes (gimbal rotation, vehicle speed).

[0177] (1) Similar to the vehicle detection module, various strategies such as training data augmentation (Gaussian blur, sharpening) are used to improve the richness of the data.

[0178] (2) Improve the network's ability to model blurred images by introducing the SR (super solution) module. For example... Figure 9 As shown, the OSNet model connects the SR module between the full-scale feature learning module and the output layer. After the input image is input into the OSNet model, feature extraction is performed first to obtain multi-scale feature maps. These multi-scale feature maps are then fused and input into the SR module to improve the resolution of the fused feature map.

[0179] (iii) Vehicle following module.

[0180] Overall, the vehicle following logic in this example is improved in the following aspects:

[0181] (1) You only need to follow one prominent target and the camera will rotate with that target. This involves the issue of selecting a prominent target.

[0182] (2) The strategies used for ID matching are different for stationary vehicles and moving vehicles. The rotation of the camera makes the position of the detection box obtained by vehicle detection almost invalid. Therefore, during the rotation of the camera, only the reid feature can be used for matching stationary vehicles. For the selected target to follow, since the camera follows its movement, the IOU feature still has a certain effect. The matching method that combines the IOU feature and the reid feature is the ID matching method used in this example.

[0183] (3) The state update strategies of stationary vehicles and moving vehicles are different during the following process: stationary vehicles are updated by reid features, while moving vehicles are updated by both IOU features and reid features.

[0184] (4) After one follow-up, the waiting time for selecting a stationary vehicle and a moving vehicle as the next follow-up target is different. A 3-second waiting time is used for stationary vehicles, which effectively avoids the loss of the follow-up target (obstruction, blurring) and allows for a quick switch to a stationary vehicle. At the same time, considering that the process of a stationary vehicle in the picture going from still to moving takes much longer than that of a moving vehicle that suddenly appears in the picture, a 1-second switching time is used for a new moving vehicle that appears in the picture. This ensures that the follow-up target can be switched in time, thereby improving the overall follow-up effect.

[0185] (5) The tracking of stationary vehicles is long-lasting. Because the frame of view is limited when the camera rotates, saving the REID features of all stationary vehicles in the frame will help with the REID feature matching of the vehicles. In addition, to prevent new targets from switching to stationary vehicles, a template REID feature can be saved for each target. The lifespan of this feature is longer than the lifespan of the target. That is, the template REID feature will not be discarded when the target is lost due to the rotation of the frame.

[0186] The overall following process of the vehicle following solution includes:

[0187] 1. Select the target to follow.

[0188] (1) Selection rule: Select the moving vehicle with the largest area as the target to follow.

[0189] (2) If there are stationary vehicles S1 and S2 and moving vehicles D1 and D2 in the picture, and the area of ​​D1 is larger than that of D2, then the target selection order is: the first selection order is D1, and the second selection order is D2. Because the area of ​​D1 is larger than that of D2, the more prominent target is prioritized.

[0190] A moving vehicle is determined in the following way:

[0191] (1) Triggered the limit of movement distance (high sensitivity, 44 pixels / frame), that is, the inter-frame movement distance reached 44 pixels / frame.

[0192] (2) Not within the static loss time window. A time window (corresponding to the second preset duration threshold mentioned above) will be given before the static vehicle switches. If the target stops moving again after the window, it is considered that there is no target and no movement.

[0193] Follow state initialization:

[0194] Stationary vehicles S1 and S2: Record frame position and size, REID features, loss duration, etc.

[0195] Dynamic vehicles D1 and D2: Record the position and size of the bounding box, reid features, and loss duration, etc. D1 should be labeled as the current target being followed.

[0196] 2. Control the gimbal rotation according to the movement of the target being followed.

[0197] Rotation logic: Because D1 was selected, the upward throw angle of D1 drives the gimbal to rotate.

[0198] Follow status updates:

[0199] (1) First, the gimbal starts to rotate.

[0200] (2) Update the position and size of the bounding box D1 for moving vehicles, update the reid feature, and update the loss duration. Moving vehicles mainly use IOU matching and secondarily reid feature matching.

[0201] (3) Update the position and size of the D2 frame of the moving vehicle, update the reid feature, update the lost time, etc.

[0202] (4) Stationary vehicles S1 and S2: The position and size of the cleared vehicles are not updated during the rotation process (i.e., the detection box of the stationary vehicle that has disappeared from the current frame does not need to be updated, when the reid feature of the stationary vehicle can be recorded), reid feature update, loss duration update, etc. During this process, the following of stationary vehicles can only be achieved through reid feature matching.

[0203] (5) New vehicles will only start recording their status after the target disappears (including the disappearance of D1 and D2). This determines the following order as follows: first follow all the moving vehicles in the initial screen, and then follow the newly added moving vehicles.

[0204] 3. Follow ends or the target is lost.

[0205] For example, if D1 drives out of the frame, leaving D2, S1, and S2, and a new vehicle N1 is added due to the rotation of the gimbal, the status will be updated accordingly:

[0206] (1) Accumulate the count of lost vehicles D1.

[0207] (2) Stationary vehicles S1 and S2 begin to update their status (position and size, reid feature update, loss duration update, etc.) after D1 is lost for t (3s for static targets and 1s for dynamic targets).

[0208] (3) The new vehicle N1 is lost in D1 for t (3s for static targets and 1s for dynamic targets) and its status is officially updated (position and size, reid feature update, loss duration update, etc.).

[0209] 4. Redefine the target to follow.

[0210] The selection order is as follows: first select vehicle D2, then select the newly added vehicle N1.

[0211] As can be seen, this example provides a single-target tracking solution applicable to complex scenarios, achieving multi-scene coverage, low computational requirements, and improved tracking robustness under camera rotation. It provides a multimodal data mining workflow and lightweight object detection model techniques. Furthermore, it offers personalized modeling methods for stationary and moving objects, including state recording, IOU and reid feature matching methods during the tracking process, and different switching times for each object.

[0212] This example effectively supports model training through a complete data mining pipeline, enabling the algorithm to cover multiple scenarios. Modeling both stationary and moving vehicles separately effectively avoids switching between stationary and following targets, resulting in a better following experience. By exposing target selection interfaces (allowing users to customize target selection rules) and waiting time parameters (allowing users to customize the waiting time when switching targets), it is possible to adapt to different scenarios, providing a more flexible following experience.

[0213] In the technical solution of this application, the acquisition, storage, use, processing, transmission, provision and disclosure of video frames are all carried out with the user's authorization.

[0214] Corresponding to the above method embodiments, this application also provides a control device for an image acquisition device, such as... Figure 10 As shown, the device includes:

[0215] The detection module 1010 is used to detect the detection boxes of each specified object contained in the first video frame currently acquired by the image acquisition device, and use them as the first detection boxes;

[0216] The first determining module 1020 is used to determine, for each first detection box, if there is a stationary specified object in the specified object detected based on historical video frames that matches the REID feature of the first detection box, that the specified object in the first detection box and the matching stationary specified object are the same object.

[0217] The second determining module 1030 is used to determine that the specified object in the first detection box and the matching specified object are the same object if there is a motion specified object in the detected specified object that matches the reid feature and pixel position of the first detection box.

[0218] The third determining module 1040 is used to determine the first detection box of the specified object to be followed from the first video frame;

[0219] The control module 1050 is used to control the pose of the image acquisition device according to the position of the first detection box of the specified object to be followed, so that the specified object to be followed is located at a preset position in the acquired video frame.

[0220] Optionally, the device further includes:

[0221] The accumulation module is used to accumulate the loss duration of the latest followed specified object if there is no latest followed specified object in the first video frame before the third determining module 1040 executes the first detection box to determine the specified object to be followed from the first video frame.

[0222] The third determining module 1040 is specifically used for: if the accumulated loss time reaches the first preset time threshold, selecting the first detection box containing the specified moving object and having the largest area as the first detection box of the specified object to be followed.

[0223] Optionally, the device further includes:

[0224] The selection module is used to select the first detection box with the largest area as the first detection box of the specified object to be followed if the accumulated loss time reaches the second preset time threshold; wherein the second preset time threshold is greater than the first preset time threshold.

[0225] Optionally, the second determining module 1030 includes:

[0226] The acquisition submodule is used to acquire the predicted pixel position of each detected motion-specified object in the first video frame.

[0227] The first determining submodule is used to determine that the specified object in the first detection box and the specified object in the motion are the same object if the difference between the predicted pixel position corresponding to the specified motion object and the pixel position of the first detection box is less than a preset threshold, and the re-identification feature of the specified motion object matches the re-identification feature of the detection box.

[0228] Optionally, the detection boxes of each specified object contained in the currently acquired video frame are obtained by using a pre-trained specified object detection model; the specified object detection model is trained using a first specified sample image;

[0229] The device further includes:

[0230] The query data determination module is used to obtain query reference data based on scene images associated with a specified scene where the image acquisition device is located; wherein, the query reference data includes: descriptive text for describing a specified object in the specified scene, and / or, a template image containing the specified object in the specified scene;

[0231] The retrieval module is used to determine images from the sample images that match the features of the query reference data, thereby obtaining a first specified sample image.

[0232] Optionally, the retrieval module includes:

[0233] The second determining submodule is used to determine an image from the sample image that matches the features of the query reference data, thereby obtaining a second specified sample image; the image enhancement submodule is used to perform image enhancement processing on the obtained second specified sample image, and combine the second specified sample image and the image-enhanced image to obtain the first specified sample image;

[0234] or,

[0235] The second determining submodule is used to determine images from the sample images that match the features of the query reference data, thereby obtaining second specified sample images; the scoring submodule is used to score the second specified sample images according to a preset scoring dimension, thereby obtaining a quality score for each second specified sample image; and selects a preset number of second specified sample images with the highest quality scores as first specified sample images; the preset scoring dimension includes at least one of the following parameters of the specified objects contained in the scene image: completeness, size, and orientation;

[0236] or,

[0237] The second determination submodule is used to determine images from the sample images that match the features of the query reference data, thereby obtaining a second specified sample image; the image enhancement submodule is used to perform image enhancement processing on the obtained second specified sample image, and combine the second specified sample image and the image-enhanced image to obtain a third specified sample image; the scoring submodule is used to score the third specified sample images according to a preset scoring dimension, thereby obtaining a quality score for each third specified sample image; and selects a preset number of third specified sample images with the highest quality scores as the first specified sample images; the preset scoring dimension includes at least one of the following parameters of the specified object contained in the scene image: completeness, size, and orientation;

[0238] or,

[0239] The second determining submodule determines images from the sample images that match the features of the query reference data, thus obtaining second specified sample images; the scoring submodule is used to score the second specified sample images according to a preset scoring dimension, thus obtaining a quality score for each second specified sample image; a preset number of second specified sample images with the highest quality scores are selected as fourth specified sample images; the preset scoring dimension includes at least one of the following parameters of the specified object contained in the scene image: completeness, size, and orientation; the image enhancement submodule is used to perform image enhancement processing on the obtained fourth specified sample images, and combine the second specified sample images and the image-enhanced image to obtain a first specified sample image.

[0240] Optionally, an initial model for detecting the specified object is obtained by performing model tuning on the first specified YOLOv5 model using at least one of the following modules:

[0241] The first replacement module is used to replace the convolutional layers in the backbone network with depthwise separable convolutional layers.

[0242] The parameter pruning module is used to reduce the number of parameters in the Feature Pyramid Network (FPN).

[0243] The downsampling module is used to add a 64x downsampling layer to the backbone network;

[0244] The second replacement module is used to replace the C3 layer in the backbone network with the SEC3 layer.

[0245] The enlargement module is used to increase the size of the convolution kernel in the first convolutional layer.

[0246] Optionally, the pre-trained specified object detection model is obtained by training a second specified YOLOV5 model using the model after model adjustment as the teacher model; the number of parameters of the second specified YOLOV5 model is less than the number of parameters of the first specified YOLOV5 model.

[0247] Optionally, any re-identification feature is obtained by feature extraction using a re-identification model; the re-identification model is trained using a second specified sample image;

[0248] The device further includes:

[0249] The selection module is used to select multiple sample video frame sequences based on a high and low threshold strategy.

[0250] The fourth determining module is used to obtain a second specified sample image based on the selected sample video frame sequence; wherein the similarity between image features of the same specified object contained in any of the selected sample video frame sequences is greater than a preset similarity threshold.

[0251] Optionally, the fourth determining module is specifically used to: perform image enhancement processing on the blurred images in the selected sample video frame sequence, and use the image after image enhancement processing as the second specified sample image.

[0252] Optionally, the re-identification model is obtained by connecting an SR module between the full-scale feature learning module and the output layer of the OSNet model.

[0253] This application also provides an electronic device, such as... Figure 11 As shown, it includes:

[0254] Memory 1101 is used to store computer programs;

[0255] The processor 1102, when executing the program stored in the memory 1101, implements the steps of the control method for any of the above-mentioned image acquisition devices.

[0256] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 1102, the communication interface, and the memory 1101 communicating with each other via the communication bus.

[0257] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0258] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0259] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0260] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0261] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of the control method for any of the above-described image acquisition devices.

[0262] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the control method of any of the image acquisition devices described in the above embodiments.

[0263] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0264] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0265] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, and computer-readable storage media are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0266] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A control method for an image acquisition device, characterized in that, The method includes: The detection bounding boxes of each specified object contained in the first video frame currently captured by the image acquisition device are used as the first detection bounding boxes; For each first detection box, if there is a stationary specified object among the specified objects detected based on historical video frames that matches the re-identification features of the first detection box, then it is determined that the specified object in the first detection box and the matching stationary specified object are the same object. If among the detected specified objects, there is a motion specified object that matches both the re-identification feature and pixel position of the first detection box, then it is determined that the specified object in the first detection box and the matching motion specified object are the same object; Determine the first detection box of the specified object to be followed from the first video frame; Based on the position of the first detection box of the specified object to be followed, the pose of the image acquisition device is controlled so that the specified object to be followed is located at a preset position in the acquired video frame.

2. The method according to claim 1, characterized in that, Before determining the first detection box of the specified object to be followed from the first video frame, the method further includes: If there is no latest following specified object in the first video frame, the loss time of the latest following specified object is accumulated; Determine the first detection box of the specified object to be followed from the first video frame, including: If the accumulated loss time reaches the first preset time threshold, select the first detection box that contains the specified moving object and has the largest area as the first detection box of the specified object to be followed.

3. The method according to claim 2, characterized in that, The method further includes: If the accumulated loss time reaches the second preset time threshold, the first detection box with the largest area is selected as the first detection box of the specified object to be followed; wherein, the second preset time threshold is greater than the first preset time threshold.

4. The method according to any one of claims 1-3, characterized in that, If, among the detected specified objects, there exists a moving specified object that matches both the re-identification features and pixel position of the first detection box, then determining that the specified object in the first detection box and the matching moving specified object are the same object includes: For each detected motion-specific object, obtain the predicted pixel position of the motion-specific object in the first video frame; If the difference between the predicted pixel position corresponding to the motion specified object and the pixel position of the first detection box is less than a preset threshold, and the re-identification feature of the motion specified object matches the re-identification feature of the detection box, then it is determined that the specified object in the first detection box and the motion specified object are the same object.

5. The method according to any one of claims 1-3, characterized in that, The detection boxes for each specified object in the currently acquired video frame are obtained using a pre-trained specified object detection model; The specified object detection model is trained using a first specified sample image; The method of obtaining the first specified sample image includes: Based on the scene images associated with the specified scene where the image acquisition device is located, query reference data is obtained; wherein, the query reference data includes: descriptive text for describing a specified object in the specified scene, and / or, a template image containing the specified object in the specified scene; The first specified sample image is obtained by identifying images from the sample images that match the features of the query reference data.

6. The method according to claim 5, characterized in that, The step of determining an image from the sample image that matches the features of the query reference data to obtain a first specified sample image includes: A second specified sample image is obtained by determining an image from the sample image that matches the features of the query reference data; The obtained second specified sample image is subjected to image enhancement processing, and the first specified sample image is obtained by combining the second specified sample image and the image enhanced image. or, A second specified sample image is obtained by determining an image from the sample image that matches the features of the query reference data; The second specified sample images are scored according to a preset scoring dimension to obtain a quality score for each second specified sample image; a preset number of second specified sample images with the highest quality scores are selected as the first specified sample images; the preset scoring dimension includes at least one of the following parameters of the specified objects contained in the scene image: completeness, size, and orientation; or, A second specified sample image is obtained by determining an image from the sample image that matches the features of the query reference data; The obtained second specified sample image is subjected to image enhancement processing, and the third specified sample image is obtained by combining the second specified sample image and the image enhanced image. The third specified sample images are scored according to a preset scoring dimension to obtain a quality score for each third specified sample image; a preset number of third specified sample images with the highest quality scores are selected as the first specified sample images; the preset scoring dimension includes at least one of the following parameters of the specified objects contained in the scene image: completeness, size, and orientation; or, A second specified sample image is obtained by determining an image from the sample image that matches the features of the query reference data; The second specified sample images are scored according to a preset scoring dimension to obtain a quality score for each second specified sample image; a preset number of second specified sample images with the highest quality scores are selected as the fourth specified sample images; the preset scoring dimension includes at least one of the following parameters of the specified objects contained in the scene image: completeness, size, and orientation; The obtained fourth specified sample image is subjected to image enhancement processing, and the second specified sample image and the image enhanced image are combined to obtain the first specified sample image.

7. The method according to claim 5, characterized in that, An initial object detection model is obtained by performing at least one of the following model adjustment processes on the first specified YOLOv5 model: Replace the convolutional layers in the backbone network with depthwise separable convolutional layers; Reduce the number of parameters in the feature pyramid network; Add a 64-fold downsampling layer to the backbone network; Replace the C3 layer in the backbone network with the SEC3 layer; Increase the size of the convolution kernel in the first convolutional layer.

8. The method according to claim 7, characterized in that, The pre-trained object detection model is obtained by training a second specified YOLOv5 model using the model after model adjustment as the teacher model; the number of parameters of the second specified YOLOv5 model is less than the number of parameters of the model after model adjustment.

9. The method according to any one of claims 1-3, characterized in that, Any re-identification feature is obtained by feature extraction using a re-identification model; The re-identification model is obtained by training a second specified sample image; The second specified sample image is obtained in the following ways: Multiple sample video frame sequences are selected based on a high and low threshold strategy; Based on the selected sample video frame sequence, a second specified sample image is obtained; wherein, the similarity between image features of the same specified object contained in any selected sample video frame sequence is greater than a preset similarity threshold.

10. The method according to claim 9, characterized in that, The step of obtaining the second specified sample image based on the selected sample video frame sequence includes: Image enhancement processing is performed on the blurred images in the selected sample video frame sequence, and the enhanced images are used as the second specified sample images.

11. The method according to claim 9, characterized in that, The re-identification model is obtained by connecting the SR module between the full-scale feature learning module and the output layer of the OSNet model.

12. A control device for an image acquisition equipment, characterized in that, The device includes: The detection module is used to detect the bounding boxes of each specified object contained in the first video frame currently captured by the image acquisition device, and use them as the first detection boxes; The first determining module is used to determine that, for each first detection box, if there is a stationary specified object among the specified objects detected based on historical video frames that matches the re-identification features of the first detection box, the specified object in the first detection box and the matching stationary specified object are the same object. The second determining module is used to determine that if there is a motion-specific object in the detected specified object that matches both the re-identification features and pixel position of the first detection box, the specified object in the first detection box and the matching motion-specific object are the same object. The third determining module is used to determine the first detection box of the specified object to be followed from the first video frame; The control module is used to control the pose of the image acquisition device according to the position of the first detection box of the specified object to be followed, so that the specified object to be followed is located at a preset position in the acquired video frame.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-11.

Images