A small animal oriented face tracking map system

Abstract

The application provides a small animal-oriented face tracking image transmission system, comprising a transmitting end, a receiving end and an animal end stimulator; the transmitting end is an animal wearing structure, comprising a miniature camera and a wireless transmitting module; the miniature camera is used for capturing face images; the wireless transmitting module establishes communication with the receiving end through a wireless image transmission link and sends face video data collected by the miniature camera to the receiving end; the receiving end comprises a wireless receiving module, a video data stream module, a data analysis module and a stimulator control module. Through the cooperative matching of the transmitting end, the receiving end and the animal end stimulator, combined with wireless image transmission, face feature analysis and multi-mode stimulation feedback, the components such as the miniature camera and the wireless transmitting module in the application are small in size and light in weight, can be adapted to various small animals, and can detect small animals and reduce the dependence on manual observation.

Description

Technical Field

[0001] This invention relates to the field of biomedical engineering and neuroscience experimental equipment, and more specifically, to a facial tracking and image transmission system for small animals. Background Technology

[0002] In research fields such as neuroscience, behavioral physiology, and drug development, small animals (such as mice) are studied... Long-term, stable, and quantitative monitoring of behavior in free-moving states, combined with external stimuli to achieve neural modulation, has significant research value. Current technologies for monitoring small animal behavior mainly rely on manual observation or holistic behavioral analysis methods based on fixed camera equipment. These methods have the following shortcomings: (1) It relies on manual observation, making it difficult to achieve long-term continuous monitoring, and the data is highly subjective; (2) Monitoring methods based on fixed camera equipment usually require animals to be in a relatively fixed or restricted shooting area, making it difficult to stably obtain fine facial behavioral information such as pupil changes, eye movements and whisker movements when small animals are moving freely. (3) Existing wearable monitoring devices are mostly designed for larger animals, and have the problems of large size and heavy weight, which are not suitable for long-term wear by small experimental animals; (4) Existing systems mostly only have data acquisition or analysis functions, lack a real-time stimulus feedback mechanism based on behavior recognition results, and it is difficult to achieve the integration of behavior monitoring and neural regulation. (5) Most of them use wired equipment, which cannot adapt to complex environments. The video data acquisition cable is tangled, restricting free movement and making it impossible to freely adjust the shooting target from multiple angles.

[0003] Therefore, there is an urgent need for lightweight facial tracking systems suitable for small animals to accurately acquire facial behavior and combine it with real-time analysis and stimulus feedback to improve the automation and data accuracy of behavioral experiments. Summary of the Invention

[0004] This invention provides a facial tracking image transmission system for small animals, including a transmitter, a receiver, and an animal stimulator; The transmitter is an animal-wearable structure, including a miniature camera and a wireless transmission module; the miniature camera is used to capture facial images; the wireless transmission module establishes communication with the receiver through a wireless image transmission link and sends the facial video data captured by the miniature camera to the receiver. The receiving end includes a wireless receiving module, a video data stream module, a data analysis module, and a stimulator control module; the wireless receiving module is communicatively connected to the wireless transmitting module and is used to receive facial video data transmitted by the transmitting end; the video data stream module is used to buffer and transmit video data to the data analysis module.

[0005] As a preferred technical solution of this application, the data analysis module has two working modes: online mode and offline mode. In offline mode, the video data stream module first stores the video file and then performs offline behavior analysis. In online mode, the data analysis module decodes the real-time video, extracts the animal's real-time facial features and analyzes its behavior, and generates feedback instructions; the stimulator control module receives the feedback instructions and drives the stimulator on the animal to perform stimulation operations.

[0006] As a preferred technical solution of this application, the transmitter also includes an adjustable support arm installed on the wireless transmission module. The end of the adjustable support arm is connected to a fixed base, and the miniature camera is installed on the fixed base. The adjustable support arm is used to adjust the shooting angle of the miniature camera.

[0007] As a preferred technical solution of this application, the transmitter also includes a micro battery, which is installed on the wireless transmitter module to provide the necessary power to the wireless transmitter module and the micro camera.

[0008] As a preferred technical solution of this application, the transmitting end also includes infrared LED beads, which are mounted on a fixed base and powered by a micro battery.

[0009] As a preferred technical solution of this application, the transmitter also includes a wearing structure for wearing, the wearing structure including a connecting terminal and a base, the base being provided with a plurality of ports for plugging into the connecting terminal.

[0010] As a preferred technical solution of this application, the animal end stimulator includes a light stimulation unit, an acoustic stimulation unit, and an electrical stimulation unit, which are used to apply multi-dimensional stimulation feedback to small animals.

[0011] As a preferred technical solution of this application, the light stimulation unit is a light source, the sound stimulation unit is a loudspeaker, and the electrical stimulation unit is an electrode.

[0012] As a preferred technical solution of this application, the data analysis module of the receiving end has a built-in animal facial feature recognition model. The animal facial feature recognition model is trained based on facial samples of multiple small animals, and realizes the recognition and quantitative analysis of key points, expression features and behavioral states of small animals' faces. The data analysis module includes a facial feature recognition model trained based on machine learning or deep learning, used to detect facial key points and extract behavioral feature parameters. The behavioral feature parameters include at least one or more of pupil changes, eye movement features, beard movement, or nose movement. The miniature camera provides both wide-angle and narrow-angle shooting modes.

[0013] As a preferred technical solution of this application, the stimulator control module triggers stimulation operation based on a preset behavior threshold or the behavior recognition result output by the data analysis module, and drives the light stimulation unit, sound stimulation unit, and electrical stimulation unit to work individually or in combination.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: This application utilizes the coordinated operation of a transmitter, receiver, and animal stimulator, combined with wireless image transmission, facial feature analysis, and multi-modal stimulation feedback. The miniature camera, wireless transmitter module, and other components in this application are small in size and lightweight, making them suitable for various small animals to wear. This allows for the detection of small animals, reducing reliance on manual observation. The wireless communication design of the wireless transmitter module allows the small animals to move freely, reducing stress caused by tangled or pulled wires, while ensuring the continuity of facial image data transmission, providing complete data support for subsequent data analysis. Attached Figure Description

[0015] Figure 1 A schematic diagram of the facial tracking and image transmission system for small animals provided in this application; Figure 2 A schematic diagram of the transmitter provided in this application; Figure 3 A schematic diagram of the port structure provided in this application; Figure 4 A schematic diagram of the structure of the fixed base provided in this application; Figure 5 A schematic diagram illustrating the use of the transmitter provided in this application; Figure 6 Images captured by the miniature camera provided in this application; Figure 7 Pupil images of the animals provided in this application, captured in a darkroom and by a miniature camera; Figure 8 Pupil images of the animals provided in this application, taken in an open box and with a miniature camera; Figure 9 The diagram showing the relationship between pupil area and frame rate provided in this application; Figure 10 The pupil region distribution map provided for this application; Figure 11 A schematic diagram of the animal-worn structural adjustment provided in this application; Figure 12 The circuit diagram of the transmitter provided in this application.

[0016] The image shows: 1. Miniature battery; 2. Wireless transmitter module; 3. Adjustable support arm; 4. Connection terminal; 5. Miniature camera; 6. Infrared LED beads; 7. Fixing base; 8. Base; 801. Port. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] It should be noted that, unless otherwise specified, the embodiments and features and technical solutions in the present invention can be combined with each other.

[0019] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0020] For an example, please refer to... Figures 1-11 A facial tracking and image transmission system for small animals includes a transmitter, a receiver, and an animal stimulator. The transmitter is an animal-wearable structure, including a miniature camera 5 and a wireless transmission module 2. The miniature camera 5 is used to capture facial images. The wireless transmission module 2 establishes communication with the receiver through a wireless image transmission link, sending the facial video data captured by the miniature camera 5 to the receiver. By designing the transmitter as an animal-wearable structure, combined with the miniature camera 5 and the wireless transmission module 2, real-time capture and wireless transmission of facial images are achieved, freeing it from the constraints of traditional wired transmission. The miniaturized design of the miniature camera 5 is adapted to the size of small animals and will not hinder their movement. The wireless communication design of the wireless transmission module 2 allows the small animals to move freely, reducing the stress response caused by tangled or pulled wires, while ensuring the continuity of facial image data transmission and providing complete data support for subsequent data analysis. The wireless transmission module 2 adopts a wireless communication design. On the one hand, it eliminates the need for wiring, simplifying the equipment installation and debugging process and improving deployment efficiency in experimental or monitoring scenarios, allowing for rapid wear and networking. On the other hand, the absence of physical wiring constraints effectively expands the activity range of small animals, more closely reflecting their natural behavior and improving the authenticity and effectiveness of the collected data. It also avoids problems such as aging, breakage, and poor contact of wiring, reducing equipment failure rates and ensuring the stability and reliability of facial image data transmission. The wireless transmission method also supports flexible multi-node networking, enabling simultaneous monitoring of multiple individuals, facilitating large-scale, batch data collection. Furthermore, wireless image transmission offers advantages such as high mobility and scalability, adapting to different experimental environments and monitoring scenarios without being limited by fixed wiring locations, further enhancing the system's applicability and flexibility. It reduces stress on small animals caused by tangled or pulled wires, while ensuring the continuity of facial image data transmission, providing complete data support for subsequent data analysis.

[0021] The receiving end includes a wireless receiving module, a video data stream module, a data analysis module, and a stimulator control module. The wireless receiving module is communicatively connected to the wireless transmitting module 2 and is used to receive facial video data transmitted by the transmitting end. The wireless transmitting module 2 and the wireless receiving module adopt a 2.4GHz wireless image transmission protocol, with a transmission distance of not less than 50 meters, strong anti-interference ability, and are suitable for monitoring scenarios where small animals have a wide range of activity. The video data stream module is used to cache and transmit video data to the data analysis module. The caching function of the video data stream module can reduce data loss caused by network fluctuations during real-time transmission and ensure the integrity of video data. At the same time, its transmission function can realize the efficient transfer of video data to the data analysis module, providing a guarantee for the data analysis module to process data quickly, reducing the impact of data transmission delay on real-time analysis results, and ensuring the timeliness of subsequent feedback command generation.

[0022] Furthermore, the data analysis module has two working modes: online and offline. In offline mode, the video data stream module first stores the video files and then performs offline behavior analysis. The reason for offline behavior analysis is that more in-depth behavior analysis can be performed offline, such as joint analysis of multiple behaviors. The online mode is mainly used for real-time closed-loop stimulation. In the online mode, the data analysis module decodes the real-time video, completes the real-time facial feature extraction and behavior analysis of the animal, and generates feedback instructions; the stimulator control module receives the feedback instructions and drives the animal-end stimulator to perform stimulation operations. Furthermore, the transmitter also includes an adjustable support arm 3 mounted on the wireless transmission module 2. The end of the adjustable support arm 3 is connected to a fixed base 7. The miniature camera 5 is mounted on the fixed base 7. The adjustable support arm 3 is used to adjust the shooting angle of the miniature camera 5. The adjustable support arm 3 can be made of tin-plated iron wire. Tin-plated iron wire can be bent at will and maintain its shape. It is not easy to spring back or break, so that the shooting position of the miniature camera 5 and the distance from the subject can be adjusted.

[0023] Furthermore, the transmitter also includes a micro battery 1, which is mounted on the wireless transmitter module 2 to provide the necessary power to the wireless transmitter module 2 and the micro camera 5.

[0024] Furthermore, the transmitter also includes an infrared LED bead 6, which is mounted on a fixed base 7 and powered by a miniature battery 1. Both the miniature camera 5 and the infrared LED bead 6 are glued to the fixed base 7. The infrared LED bead 6 provides infrared illumination when photographing the pupil. Since the supply voltage of the infrared LED bead 6 is typically 1.2-1.5V, while the miniature battery 1 is a lithium battery with a typical voltage of 3.2-3.7V, a voltage divider resistor is connected in series in the circuit between the miniature battery 1 and the infrared LED bead 6 for voltage division. Figure 12 As shown.

[0025] Furthermore, the transmitter also includes a wearing structure for use, which includes a connecting terminal 4 and a base 8. The base 8 is provided with several ports 801, which are arranged in at least two rows. The ports 801 are used to connect to the connecting terminal 4. By connecting the connecting terminal 4 to different ports 801, the position of the miniature camera 5 can be adjusted, that is, the angle can be adjusted, so that the same device can shoot from multiple angles. For example: refer to Figure 3 Pull the connecting terminal 4 to separate it from the port 801. Then rotate the remaining parts except the base 8 and the port 801 by 90 degrees, 180 degrees, or 270 degrees, and then insert the connecting terminal 4 into the corresponding position of the port 801. This achieves angle adjustment. Figure 11 As shown, it can track and photograph the animal's left eye, right eye, whiskers, nose, ears, etc.; that is, the port 801 on the base 8 is used to select the fixed installation position (such as left eye / right eye, etc.), and the adjustable support arm 3 is used to finely adjust the angle and distance.

[0026] Furthermore, the animal end stimulator includes a light stimulation unit, an acoustic stimulation unit, and an electrical stimulation unit, used to apply multi-dimensional stimulation feedback to small animals.

[0027] Furthermore, the light stimulation unit is a light source, such as a light-emitting lamp bead. It should be noted that the light stimulation unit is not an infrared LED lamp bead 6, but an additional light source. The sound stimulation unit is a speaker, and the electrical stimulation unit is an electrode. The light stimulation unit, speaker, and electrodes are all used to provide external sensory stimulation to the mouse. In addition to the above-mentioned external sensory stimulation, odor, impact, food, water rewards, etc. can also be used.

[0028] Furthermore, the data analysis module of the receiving end has a built-in animal facial feature recognition model, which is trained based on facial samples of multiple small animals, to realize the recognition and quantitative analysis of key facial points, expression features, and behavioral states of small animals. The data analysis module includes a facial feature recognition model trained based on machine learning or deep learning, used to detect facial key points and extract behavioral feature parameters. These behavioral feature parameters include at least one or more of the following: pupil changes, eye movement features, beard movement, or nose movement. Figure 6 As shown, the miniature camera 5 offers both wide-angle and narrow-angle shooting modes. Figure 6 The image on the left shows facial expressions captured with a wide-angle view. A wide-angle view can capture more facial surface information, which is helpful for facial expression analysis, while a narrow view can focus more on capturing clear information such as pupils, ears, and nose. The lens of the miniature camera 5 is a replaceable fixed-focus lens. The mounting orientation can be adjusted by inserting the connecting terminal 4 into the corresponding port 801. The object distance between the miniature camera 5 and the object being photographed can be changed by moving the adjustable support arm 3 to achieve a wide-angle or narrow field of view.

[0029] Furthermore, the stimulator control module triggers stimulation operations based on preset behavior thresholds or behavior recognition results output by the data analysis module, driving the light stimulation unit, sound stimulation unit, and electrical stimulation unit to work individually or in combination.

[0030] Taking the light-dark chamber monitoring of mice as an example, refer to Figures 7-10 , Figure 7 The left image shows a schematic diagram of a mouse inside a dark box. Figure 7 The image on the right shows the pupil of a mouse inside a dark box, captured by a miniature camera 5. Figure 8 The left image shows a schematic diagram of a mouse inside a bright box. Figure 8 The image on the right shows the pupil of a mouse inside a bright box, captured by a miniature camera 5. The light / dark box experiment is used to detect the anxiety state of mice in light / dark environments. Traditional wired devices cannot follow the mice as they move between the light and dark boxes, thus failing to monitor the mice's faces. The wireless design of this application can stably and reliably track the mice's faces within the light / dark box, which is beneficial for meeting more flexible and complex experimental needs and providing more detailed and comprehensive behavioral indicators for neuroscience experiments.

[0031] This application adopts a low-power radio image transmission scheme, and the components of the transmitter adopt a miniaturized structure, which ensures long battery life and light weight while meeting scientific research needs and has high application value. Due to its light weight, it can be used for facial tracking of small animals such as mice, hamsters, and bats for scientific research.

[0032] In use, the base 8 is fixed to the head of the small animal with dental cement, and the connecting terminal 4 is plugged into the corresponding port 801 of the base 8. The tracking position of the miniature camera 5 is determined according to the experimental requirements, such as the left eye, right eye, whiskers, etc. The shooting angle and distance of the miniature camera 5 are adjusted by the adjustable support arm 3. Start the wireless receiving module, video data stream module, data analysis module and stimulator control module, ensure that each module is operating normally, confirm that the wireless receiving module of the receiving end is matched with the wireless transmitting module 2 of the transmitting end, and establish a communication connection using the 2.4GHz wireless image transmission protocol. Depending on the experimental scenario and requirements, the corresponding working mode can be selected through the data analysis module of the receiving end. If it is necessary to observe the facial reactions of small animals in real time and apply stimulation, the online mode can be selected. At this time, the data analysis module will decode the real-time video transmitted by the miniature camera 5, automatically complete facial feature extraction and behavior analysis, and generate feedback instructions. Alternatively, the offline mode can be selected. The video data stream module will first store the video files collected by the miniature camera 5, and then perform offline behavior analysis, such as multi-behavior joint analysis. If online mode is selected, the stimulator control module will receive feedback instructions generated by the data analysis module and drive the light stimulation unit, sound stimulation unit, and electrical stimulation unit of the animal end stimulator individually or in combination, according to experimental needs. The light stimulation unit applies light stimulation to the animal's eyes, the sound stimulation unit applies sound stimulation to the animal's ears, and the electrical stimulation unit applies electrical stimulation to the animal. During the application process, the animal's condition needs to be observed to reduce overstimulation and stress response. During the experiment, the video display at the receiving end was observed in real time to confirm that the miniature camera 5 was tracking stably, the video data transmission was normal, and the data analysis module was outputting recognition results and quantitative data normally. If the offline mode was used, the storage status of the video data stream module was checked regularly to ensure that no data was lost. If the online mode was used, the corresponding data of stimulus feedback and the animal's facial reactions and behavioral states were recorded synchronously to provide support for subsequent scientific research analysis.

[0033] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0034] Obviously, the embodiments described above are merely some embodiments of the present invention, not all embodiments. The accompanying drawings show preferred embodiments of the present invention, but do not limit the patent scope of the present invention. The present invention can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the patent protection scope of this invention.

Claims

1. A facial tracking and image transmission system for small animals, characterized in that, Includes the transmitter, receiver, and animal stimulator; The transmitter is an animal-wearable structure, including a miniature camera (5) and a wireless transmission module (2); the miniature camera (5) is used to capture facial images; the wireless transmission module (2) establishes communication with the receiver through a wireless image transmission link and sends the facial video data collected by the miniature camera (5) to the receiver; the transmitter also includes an adjustable arm (3) installed on the wireless transmission module (2), the end of the adjustable arm (3) is connected to a fixed base (7), the miniature camera (5) is installed on the fixed base (7), and the adjustable arm (3) is used to adjust the shooting angle of the miniature camera (5); the transmitter also includes a wearing structure for wearing, the wearing structure includes a connecting terminal (4) and a base (8), the base (8) is provided with several ports (801), and the ports (801) are used to plug into the connecting terminal (4); The receiving end includes a wireless receiving module, a video data stream module, a data analysis module, and a stimulator control module; the wireless receiving module is communicatively connected to the wireless transmitting module (2) and is used to receive facial video data transmitted by the transmitting end; the video data stream module is used to buffer and transmit video data to the data analysis module.

2. The facial tracking and image transmission system for small animals according to claim 1, characterized in that, The data analysis module has two working modes: online mode and offline mode. In offline mode, the video data stream module first stores the video file and then performs offline behavior analysis. In online mode, the data analysis module decodes the real-time video, extracts the animal's real-time facial features and analyzes its behavior, and generates feedback instructions; the stimulator control module receives the feedback instructions and drives the stimulator on the animal to perform stimulation operations.

3. The facial tracking and image transmission system for small animals according to claim 1, characterized in that, The transmitter also includes a micro battery (1), which is mounted on the wireless transmitter module (2) to provide the necessary power to the wireless transmitter module (2) and the micro camera (5).

4. The facial tracking and image transmission system for small animals according to claim 3, characterized in that, The transmitter also includes an infrared LED bead (6), which is mounted on a fixed base (7) and is powered by a micro battery (1).

5. The facial tracking and image transmission system for small animals according to claim 1, characterized in that, The animal end stimulator includes a light stimulation unit, an acoustic stimulation unit, and an electrical stimulation unit, used to apply multi-dimensional stimulation feedback to small animals.

6. The facial tracking and image transmission system for small animals according to claim 5, characterized in that, The light stimulation unit is a light source, the sound stimulation unit is a loudspeaker, and the electrical stimulation unit is an electrode.

7. The facial tracking and image transmission system for small animals according to claim 1, characterized in that, The data analysis module of the receiving end has a built-in animal facial feature recognition model, which is trained based on facial samples of multiple small animals, and realizes the recognition and quantitative analysis of key facial points, expression features and behavioral states of small animals.

8. The facial tracking and image transmission system for small animals according to claim 7, characterized in that, The data analysis module includes a facial feature recognition model trained based on machine learning or deep learning, used to detect facial key points and extract behavioral feature parameters. The behavioral feature parameters include at least one or more of pupil changes, eye movement features, beard movement, or nose movement.

9. The facial tracking and image transmission system for small animals according to claim 1, characterized in that, The miniature camera (5) offers two shooting modes: wide-angle and narrow-angle.

10. The facial tracking and image transmission system for small animals according to claim 5, characterized in that, The stimulator control module triggers stimulation operations based on a preset behavior threshold or the behavior recognition results output by the data analysis module, driving the light stimulation unit, sound stimulation unit, and electrical stimulation unit to work individually or in combination.

Images