Active unscrambling method for observing free-moving animals based on deep learning

Abstract

The application belongs to the technical field of automatic control uncoiling, and discloses a method for actively uncoiling a free-moving animal based on deep learning, S1 continuously acquires animal image frames and pre-processes; S2 inputs the image into a deep learning model to obtain head and tail key point coordinates and confidence; S3 calculates an average confidence p; S4 calculates an animal current orientation angle according to head and tail key point vectors; S5 acquires a current motor angle, calculates an angle deviation and performs normalization; S6 judges whether it is less than a dead zone threshold, if yes, no action is performed; S7 if not, a motor is controlled to smoothly rotate an angle according to preset speed parameters; and S8 repeats S1-S7 to make the motor follow real-time changes in the animal orientation. The application proposes a sensor-free visual closed-loop control architecture, cancels physical sensors, eliminates response delay and accumulated error, and improves system robustness and control accuracy through average confidence judgment, angle normalization and dead zone threshold setting.

Description

Technical Field

[0001] This invention relates to the field of automatic control unwinding technology, and more particularly to an active unwinding method based on deep learning observation of freely moving animals. Background Technology

[0002] In neuroscience and behavioral research, recording brain activity in freely moving animals (such as mice and rats) using optical fibers or electrodes has become a routine experimental technique. Because experimental animals constantly move and turn during free exploration, the signal transmission lines (such as optical fibers and data cables) connected to the animal's head are prone to tangling, causing experimental interruptions, data loss, and even injury to the animal. To address this problem, researchers have proposed an "active untangling system," which uses a motor-controlled rotating platform to allow experimental cables to rotate in real time with the animal's movement, thus preventing tangling.

[0003] Traditional active unrotation systems rely on sensors or magnetic encoders to detect the animal's orientation or rotation angle, but these solutions generally suffer from insufficient accuracy, slow response, and sensitivity to external interference. Some systems attempt to estimate the animal's orientation through tag recognition or image processing, but the results are easily affected by occlusion and changes in lighting, and the recognition algorithms lack generalization ability. Summary of the Invention

[0004] The present invention aims to provide an active unwinding method based on deep learning for observing freely moving animals, in order to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: An active unwinding method based on deep learning for observing freely moving animals is applied to a system including an image acquisition device, a controller, a motor, and a motor-driven unwinding mechanism. The unwinding mechanism connects and unwinds the connector between the animal head-mounted device and the data acquisition unit, and includes the following steps: S1: Continuously acquire image frames containing the freely moving animal through the image acquisition device, and preprocess the image frames; S2: Input the preprocessed image frames into the pre-trained deep learning pose estimation model to infer the coordinate information and confidence level of key points of the animal's head and tail; S3: Calculate the average confidence level p of the key points of the head and tail, and determine whether the average confidence level p is greater than the preset confidence level threshold. Only when the average confidence level p is greater than the threshold will subsequent unwinding control be performed to avoid malfunctions due to single-point recognition errors. S4: Calculate the animal's current orientation angle based on the vector formed by the head key points and the tail key points. The initial angle of the image space is 0° when the animal is facing directly to the right; S5: Obtain the current rotation angle of the motor. Calculate the angle deviation and to Normalization was performed using... The formula maps it to the interval (-180°, 180°), eliminating ambiguity in direction judgment caused by periodic changes in angle; S6: Determine the absolute value of the normalized angle deviation. Is it less than the preset dead zone threshold? ,like If the animal's slight tremors cause the motor to start and stop frequently, the unwinding action will not be performed. S7: If The controller then controls the motor to generate a smooth motion pulse sequence with preset acceleration, deceleration, initial speed, and maximum speed parameters, causing the motor to rotate according to an S-shaped or trapezoidal speed curve. Angle, to achieve flexible unspinning of the connector; S8: Repeat steps 1 to 7 to make the rotation angle of the motor follow the animal's orientation angle in real time.

[0006] Preferably, the gear transmission assembly includes a driving gear and a driven gear that mesh with each other. The driving gear is connected to the output shaft of the stepper motor, the driven gear is connected to the rotor of the conductive slip ring, and the data acquisition unit is fixedly connected to the driven gear. The stepper motor rotates in the opposite direction to the animal's spin. After gear transmission, the driven gear drives the data acquisition device to rotate in the same direction as the animal's spin, thus achieving mechanical unspinning and matching the direction of the animal's movement.

[0007] Preferably, the deep learning pose estimation model is any one of DeepLabCut, LEAP, or OpenPose. The model adapts to different experimental animals or different image acquisition perspectives through pre-trained weight files without changing the mechanical structure of the unwinding mechanism.

[0008] Preferably, the image acquisition device continuously acquires image frames at a fixed frame rate and adopts a frame-synchronized closed-loop control architecture: the host establishes a producer-consumer queue, the image acquisition thread acts as a producer to put image frames into the queue, and the deep learning inference thread acts as a consumer to obtain image frames from the queue for processing. A "latest frame priority" strategy is adopted, and when there are multiple unprocessed image frames in the queue, the latest frame is processed first and expired frames are discarded to reduce the control delay caused by inference blocking and ensure the real-time performance of the visual closed loop.

[0009] Preferably, before starting S1, an initial calibration step is also included: before the animal begins free movement, the motor is calibrated to zero to adjust the initial angle of the motor. The initial orientation angle of the animal in image space Maintain consistency and establish a unified coordinate benchmark.

[0010] Preferably, in S5, the angle deviation is... During normalization, the reduction ratio of the gear transmission assembly and the microstepping coefficient of the stepper motor are also considered to normalize the rotation angle. This is converted into the target number of pulses for the stepper motor, enabling precise angle positioning.

[0011] Preferably, the preset dead zone threshold in S6 The adjustment is dynamically based on the physiological shaking characteristics of the experimental animals and the flexibility of the connectors.

[0012] Preferably, in step S7, when controlling the motor rotation, a position detection mechanism is also incorporated. Once the motor has rotated to the correct position, the motor angle is read again. and the current animal orientation angle If the deviation is still greater than the dead zone threshold, a second fine-tuning is performed until the deviation is eliminated.

[0013] Preferably, the system also includes an anomaly handling step: when the average confidence level p of multiple consecutive frames of images is lower than the confidence level threshold, or when a motor stall or limit switch triggering anomaly is detected, the system pauses the automatic unwinding mode and issues an alarm signal, or switches to the manual unwinding interface, allowing the experimenter to manually control the unwinding.

[0014] Preferably, the connector is one or more of FPC flexible cable, optical fiber or bioelectric signal transmission line. The unwinding control is protected by a smooth motion curve and a dead zone threshold to avoid mechanical damage to the connector and extend its service life.

[0015] The beneficial effects of this technical solution compared to existing technologies are as follows: (1) This invention proposes a sensorless visual closed-loop control architecture based on deep learning, which changes the traditional technical path of active unrotation systems that rely on physical sensors (such as angle encoders, magnetic encoders, or tension sensors). Traditional solutions require integrating sensors into the rotating mechanism, which not only increases system complexity and hardware cost, but also faces inherent defects such as sensor hysteresis response, wiring interference, and cumulative errors. This invention acquires animal images in real time through a top camera, and directly identifies key points of the animal's head and tail by combining them with a deep learning posture estimation model, calculates their orientation angle, and uses this as the sole control source to drive the motor to follow the rotation, forming a complete closed loop of "visual perception → angle calculation → motor control → feedback calibration". This architecture eliminates physical sensors and their auxiliary circuits, simplifies the system structure, and eliminates sensor response delay and cumulative errors, realizing real-time following unrotation.

[0016] (2) This invention avoids motor malfunction caused by single-frame recognition errors by calculating the average confidence of key points at the head and tail and setting a threshold; it normalizes the angle deviation to eliminate ambiguity in direction judgment caused by periodic changes in angle (such as sudden changes from 359° to 0°); and it introduces a dynamically adjustable dead zone threshold to achieve a balance between unwinding accuracy and motor start-stop frequency, avoiding frequent start-stop caused by slight animal tremors.

[0017] (3) The present invention generates a smooth motion curve by setting parameters such as acceleration and deceleration, so that the motor can start and stop flexibly according to the S-shaped or trapezoidal speed curve, avoiding impact damage to fragile connectors such as FPC flexible wires and optical fibers. Attached Figure Description

[0018] Figure 1 A flowchart provided for this invention; Figure 2 The method steps provided by this invention are illustrated. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments:

[0020] Example 1

[0021] This embodiment provides an active unwinding method for observing freely moving animals based on deep learning, applied to a system including an image acquisition device, a controller, a motor, and an unwinding mechanism driven by the motor. The unwinding mechanism is used to connect and unwind the connector between the animal head-mounted device and the data acquisition device. In this embodiment, the animal is a mouse, and the connector is an FPC flexible cable.

[0022] The unwinding mechanism includes a test frame, a top plate, a bottom plate, a stepper motor, a drive assembly, a data acquisition device, a conductive slip ring, a support frame, a connecting rod, and a controller. The top plate is fixed to the top of the test frame, and the bottom plate is fixed to the bottom of the test frame, forming a free movement space for the mouse. The stepper motor is mounted on the upper surface of the top plate, with its output shaft passing through the top plate. The conductive slip ring is mounted on the upper surface of the top plate, with its rotor passing through the top plate. The drive assembly includes a meshing drive gear and a driven gear. The drive gear is mounted on the output shaft of the stepper motor, and the driven gear is connected to the rotor of the conductive slip ring. The support frame is fixed to the tooth surface of the driven gear facing the movement space, and the data acquisition device is fixedly connected to the other end of the support frame. The connecting rod is fixed to the tooth surface of the driven gear facing the movement space and adjacent to the support frame; the connecting rod is L-shaped. An image acquisition device (an industrial camera in this embodiment) is arranged above the test space to acquire real-time images of the mouse's movements. The controller is fixedly mounted on the upper surface of the top plate and electrically connected to the stepper motor and the image acquisition device.

[0023] Based on the above hardware system, the active unspinning method in this embodiment includes the following steps: S1: Image frames containing freely moving mice are continuously acquired using an industrial camera positioned above the test space. The sampling frame rate of the industrial camera is set to 30fps to ensure real-time capture of the mice's movement changes.

[0024] Each captured image frame undergoes preprocessing, specifically including: (1) Color space conversion: Since the images captured by the industrial camera are in BGR format, while the subsequent deep learning model is trained based on RGB format, it is necessary to convert the image frames from BGR color space to RGB color space; (2) Region of Interest (ROI) cropping: Based on the actual range of the test space, the image frame is cropped to remove irrelevant background areas and reduce the computational load of subsequent processing; (3) Data type conversion: Convert the image data to uint8 data type to meet the input requirements of the deep learning model.

[0025] S2: Input the preprocessed image frames into a pre-trained deep learning pose estimation model. In this embodiment, DeepLabCut is used as the deep learning model, and ResNet-50 is used as the backbone network. The model is pre-trained on a mouse dataset and can accurately identify key points on the mouse head and tail.

[0026] The model inference outputs the coordinates and confidence scores of key points on the mouse's head and tail. It should be noted that the DeepLabCut model outputs coordinates in (y, x) format, while subsequent angle calculations require (x, y) format. Therefore, this step also includes converting the coordinate information from (y, x) format to (x, y) format.

[0027] S3: Calculate the average confidence level p = (p_head + p_tail) / 2 for the two key points at the head and tail, and determine whether the average confidence level p is greater than the preset confidence level threshold (the threshold is set to 0.8 in this embodiment).

[0028] Subsequent unwinding control is only performed when the average confidence level p is greater than the threshold to avoid motor malfunction due to single-point recognition errors. If the average confidence level p is less than or equal to the threshold, the angle calculation and unwinding control steps for the current frame are skipped, and the process directly returns to S1 to obtain the next frame image for processing.

[0029] S4: Calculate the mouse's current orientation angle θ_img by constructing a vector from the head keypoint coordinates (x_head, y_head) and tail keypoint coordinates (x_tail, y_tail). The specific calculation formula is as follows: ; The image space starts at 0° with the mouse facing directly to the right, and the angle range is defined as (-180°, 180°).

[0030] S5: Read the current rotation angle of the stepper motor via the controller. In this embodiment, the stepper motor is equipped with an encoder, which can directly read the absolute angle; if a stepper motor without an encoder is used, the current angle can also be calculated based on the number of drive pulses.

[0031] Calculate angular deviation .

[0032] Because the angle is periodic, direct use This may lead to incorrect direction judgment (e.g., when...) It is 1°. At 359°, = -358°, should actually be rotated +2°). Therefore, it is necessary to adjust... After normalization, it is mapped to the interval (-180°, 180°) using the following formula: ; Here, mod represents the modulo operation.

[0033] By normalizing the process, ambiguity in direction determination caused by periodic changes in angle is eliminated, ensuring that the motor always rotates to the target angle along the shortest path.

[0034] In this step, the reduction ratio of the gear transmission assembly and the microstepping coefficient of the stepper motor are also combined to determine the rotation angle. This is converted into the target number of pulses for the stepper motor. In this embodiment, the reduction ratio between the driving gear and the driven gear is 1:2, and the stepper motor microstepping factor is set to 16. The controller calculates the required number of pulses based on these parameters.

[0035] S6: Set dead zone threshold In this embodiment, based on the physiological shaking characteristics of mice and the flexibility of the FPC wire, Dynamically adjust to 3°. Determine the absolute value of the normalized angle deviation. Is it less than .

[0036] like This indicates that the current angle deviation is small and insufficient to form an entanglement trend. To avoid frequent motor start-stop due to the mouse's slight shaking, the unwinding action is not performed, and the system directly returns to S1 to obtain the next frame image.

[0037] like Then, enter S7 to perform unspin control.

[0038] S7: Based on the target pulse count calculated in S5 and the preset motion control parameters, the controller generates a smooth motion pulse sequence to drive the stepper motor to rotate. In this embodiment, the parameters are set as follows: acceleration 1000 pulse / s², deceleration 1000 pulse / s², initial speed 100 pulse / s, and maximum speed 2000 pulse / s.

[0039] The stepper motor rotates according to a trapezoidal speed curve The angle is adjusted to achieve flexible unwinding of the FPC wire, avoiding impact damage. The stepper motor rotates in the opposite direction to the mouse's spin. After gear transmission, the driven gear drives the data acquisition device to rotate in the same direction as the mouse's spin, thus achieving mechanical unwinding and matching the direction of animal movement.

[0040] This step also incorporates a positioning detection mechanism; once the motor has rotated to the correct position, the motor angle is read again. and the current animal orientation angle If the deviation is still greater than the dead zone threshold, a second fine-tuning is performed until the deviation is eliminated.

[0041] S8: Repeat steps S1 to S7 to make the motor's rotation angle follow the mouse's orientation angle in real time, achieving active unrotation. The industrial camera continuously acquires images at a frame rate of 30fps, and the control loop is executed synchronously with the image acquisition frame rate, forming a visual closed-loop control of "acquisition → inference → control → feedback".

[0042] To further improve real-time performance, this embodiment employs an optimized frame synchronization control architecture: a producer-consumer queue is established on the host side. The image acquisition thread acts as the producer, adding image frames to the queue, while the deep learning inference thread acts as the consumer, retrieving image frames from the queue for processing. When multiple unprocessed images exist in the queue, a "latest frame priority" strategy is adopted, prioritizing the processing of the latest frame and discarding expired frames. This reduces control latency caused by inference blocking and ensures the real-time performance of the visual closed loop.

[0043] Before starting S1, i.e., after the animal is placed in the test space but before it begins free movement, an initial calibration procedure is performed. The controller controls the stepper motor to return to zero, bringing the motor's initial angle... The initial orientation angle of the animal in image space To maintain consistency, an image frame is acquired using an image acquisition device. A deep learning model is then used to identify key points on the mouse's head and tail, and their orientation angles are calculated. Then, drive the motor to rotate to the position corresponding to that angle, and establish a unified coordinate reference.

[0044] Throughout the experiment, the system monitors its operational status in real time. When the average confidence level p of multiple consecutive frames falls below the confidence threshold, or when abnormalities such as motor stall or limit switch triggering are detected, the system pauses the automatic unwinding mode and issues an alarm signal. Simultaneously, the system provides a manual unwinding interface, allowing experimenters to switch to manual mode and manually control the motor rotation for unwinding via the controller.

[0045] In this embodiment, the connector is an FPC flexible cable. The dead-zone threshold judgment in S6 and the smooth motion curve in S7 provide dual protection, preventing mechanical damage to the FPC flexible cable and extending its service life. This method is also applicable to other types of connectors such as optical fibers and bioelectric signal transmission lines.

[0046] Example 2

[0047] The difference between this embodiment and Embodiment 1 is that: (1) The deep learning pose estimation model uses LEAP to identify key points on the head and tail of rats; (2) Dead zone threshold The angle was dynamically adjusted to 5° based on the physiological shaking characteristics of rats. (3) The connector is an optical fiber, used for optical signal transmission in optogenetic experiments; (4) The motor motion control adopts an S-shaped speed curve to further reduce start-stop impact.

[0048] By changing the pre-training weights, this method can be adapted to different experimental animals (mice, rats) and different image acquisition perspectives (top view, tilt view) without changing the mechanical structure of the unwinding mechanism, thus exhibiting good versatility.

[0049] The above descriptions are merely embodiments of the present invention, and common knowledge such as specific technical solutions and / or characteristics are not described in detail here. It should be noted that those skilled in the art can make various modifications and improvements without departing from the technical solutions of the present invention, and these should also be considered within the scope of protection of the present invention. These modifications and improvements will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. An active unwinding method based on deep learning for observing freely moving animals, applied to a system including an image acquisition device, a controller, a motor, and an unwinding mechanism driven by the motor, wherein the unwinding mechanism is used to connect and unwind the connector between the animal head-mounted device and the data acquisition device, characterized in that... Includes the following steps: S1: Continuously acquire image frames containing the freely moving animal through the image acquisition device, and preprocess the image frames; S2: Input the preprocessed image frames into the pre-trained deep learning pose estimation model to infer the coordinate information and confidence level of key points of the animal's head and tail; S3: Calculate the average confidence level p of the key points of the head and tail, and determine whether the average confidence level p is greater than the preset confidence level threshold. Only when the average confidence level p is greater than the threshold will subsequent unwinding control be performed to avoid malfunctions due to single-point recognition errors. S4: Calculate the animal's current orientation angle based on the vector formed by the head key points and the tail key points. The initial angle of the image space is 0° when the animal is facing directly to the right; S5: Obtain the current rotation angle of the motor. Calculate the angle deviation and to Normalization was performed using... The formula maps it to the interval (-180°, 180°), eliminating ambiguity in direction judgment caused by periodic changes in angle; S6: Determine the absolute value of the normalized angle deviation. Is it less than the preset dead zone threshold? ,like If the animal's slight tremors cause the motor to start and stop frequently, the unwinding action will not be performed. S7: If The controller then controls the motor to generate a smooth motion pulse sequence with preset acceleration, deceleration, initial speed, and maximum speed parameters, causing the motor to rotate according to an S-shaped or trapezoidal speed curve. Angle, to achieve flexible unspinning of the connector; S8: Repeat steps 1 to 7 to make the rotation angle of the motor follow the animal's orientation angle in real time.

2. The active unwinding method for observing freely moving animals based on deep learning as described in claim 1, characterized in that: The gear transmission assembly includes a driving gear and a driven gear that mesh with each other. The driving gear is connected to the output shaft of the stepper motor, the driven gear is connected to the rotor of the conductive slip ring, and the data acquisition unit is fixedly connected to the driven gear. The stepper motor rotates in the opposite direction to the animal's spin. After gear transmission, the driven gear drives the data acquisition device to rotate in the same direction as the animal's spin, thus achieving mechanical unspinning and matching the direction of the animal's movement.

3. The active unwinding method for observing freely moving animals based on deep learning as described in claim 1, characterized in that: The deep learning pose estimation model can be any one of DeepLabCut, LEAP, or OpenPose. The model adapts to different experimental animals or different image acquisition perspectives through pre-trained weight files without changing the mechanical structure of the unwinding mechanism.

4. The active unwinding method for observing freely moving animals based on deep learning as described in claim 1, characterized in that: The image acquisition device continuously acquires image frames at a fixed frame rate and adopts a frame-synchronized closed-loop control architecture: the host establishes a producer-consumer queue, the image acquisition thread acts as the producer to put image frames into the queue, and the deep learning inference thread acts as the consumer to retrieve image frames from the queue for processing. A "latest frame priority" strategy is adopted, when there are multiple unprocessed image frames in the queue, the latest frame is processed first and expired frames are discarded to reduce the control delay caused by inference blocking and ensure the real-time performance of the visual closed loop.

5. The active unwinding method for observing freely moving animals based on deep learning as described in claim 1, characterized in that: Before starting S1, an initial calibration step is also included: before the animal begins free movement, the motor is calibrated to zero to ensure the motor's initial angle is correct. The initial orientation angle of the animal in image space Maintain consistency and establish a unified coordinate benchmark.

6. The active unwinding method for observing freely moving animals based on deep learning as described in claim 1, characterized in that: S5 for angle deviation During normalization, the reduction ratio of the gear transmission assembly and the microstepping coefficient of the stepper motor are also considered to normalize the rotation angle. This is converted into the target number of pulses for the stepper motor, enabling precise angle positioning.

7. The active unwinding method for observing freely moving animals based on deep learning as described in claim 1, characterized in that: The dead zone threshold preset in S6 The adjustment is dynamically based on the physiological shaking characteristics of the experimental animals and the flexibility of the connectors.

8. The active unwinding method for observing freely moving animals based on deep learning as described in claim 1, characterized in that: When controlling the motor rotation in S7, a position detection mechanism is also incorporated. Once the motor has rotated to the correct position, the motor angle is read again. and the current animal orientation angle If the deviation is still greater than the dead zone threshold, a second fine-tuning is performed until the deviation is eliminated.

9. The active unwinding method for observing freely moving animals based on deep learning as described in claim 1, characterized in that: It also includes anomaly handling steps: when the average confidence level p of multiple consecutive frames of images is lower than the confidence level threshold, or when a motor stall or limit switch triggering anomaly is detected, the system pauses the automatic unwinding mode and issues an alarm signal, or switches to the manual unwinding interface, whereby the experimenter manually controls the unwinding.

10. The active unwinding method for observing freely moving animals based on deep learning as described in claim 1, characterized in that: The connector is one or more of FPC flexible wire, optical fiber or bioelectric signal transmission line. The unwinding control uses a smooth motion curve and dead zone threshold for dual protection to avoid mechanical damage to the connector and extend its service life.

Images