A dual-modal visual behavior monitoring system for intelligent agents
Through the dual-mode visual behavior monitoring system of the intelligent body, the LCD screen and camera combined with deep learning algorithms are used to realize multi-functional monitoring of visual dynamic reaction experiments and light-dark shuttle experiments, solving the problem of single device functions and high cost in the existing technology. The problem of low data accuracy achieves low-cost and accurate dual-modal experimental analysis.
Patent Information
- Application Number
- CN202010220166.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-03-25
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-03-25
AI Technical Summary
The existing visual dynamic reaction experiment and light-dark shuttle experiment devices have single functions, high cost, large equipment, inaccurate image recognition, and low accuracy of analysis data, so two experiments cannot be performed at the same time.
Design a dual-modal visual behavior monitoring system for intelligent bodies, including a detachable mirror panel and a fixed platform, combined with an LCD screen and a camera, and realize multifunctional monitoring of mouse behavior through binary processing and deep learning algorithms. analyze.
It realizes a low-cost, small space and accurate analysis dual-modal experimental device, which can operate independently, saves manpower, material resources and time resources, has reliable data, is suitable for widespread applications, and provides strong support for basic scientific research in ophthalmology.
Smart Images

Figure CN111445483B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to multiple technical fields such as ophthalmology, image recognition, and automatic control, and provides a composite device suitable for mouse visual kinematic experiments and light-dark shuttle experiments and a monitoring method using the device. Specifically, it relates to an intelligent dual-modal visual behavior monitoring system and a monitoring method. Background Art
[0002] In basic research in ophthalmology, when model animals such as disease model mice are used to study the pathogenic mechanism of specific diseases, the evaluation of mouse visual function is very important. Since animals cannot describe their subjective feelings and there are large differences between different individuals, the evaluation of the visual function of model animals has always been one of the difficult problems in the field of ophthalmology research. At present, the visual motion response test and the light-dark shuttle test are mostly used internationally to evaluate the visual function of mice.
[0003] Mice have an autonomous following, resetting, and following process for moving objects. Based on this biological phenomenon, monitoring and recording the behavior of mice in the moving grating to evaluate visual function is called a visual motion reaction experiment. In addition, mice have an instinctive habit of moving toward darkness and avoiding light. Putting mice in a device that connects light and dark, monitoring the number of times the mice shuttle through it and the time they stay in the light box to evaluate their visual function is called a light-dark shuttle experiment. There are existing visual motion reaction devices and light-dark shuttle experimental devices used at home and abroad, but their functions are relatively simple and can only perform one of the tests. In addition, most of them have defects such as high cost, large equipment, and inaccurate image recognition. At the same time, because their algorithms are relatively fixed, there is often a situation where the accuracy of the analysis data is poor. Since these two experiments have certain commonalities in the device requirements, if the same set of devices can be used to complete the above two experiments, the experimental costs will be saved to a certain extent. Summary of the invention
[0004] The present invention provides an intelligent agent dual-modal visual behavior monitoring system and a monitoring method, which are used to solve at least one problem existing in the above-mentioned prior art.
[0005] To achieve the above object, the present invention provides an intelligent dual-modal visual behavior monitoring system, which comprises: a first compartment and a second compartment,
[0006] The tops of the first compartment and the second compartment are both openable to allow mice to be placed in and taken out for experiments.
[0007] The inner wall of the first compartment is made of a light-proof material.
[0008] The second compartment has an inner wall and an outer wall, the outer wall is made of an opaque material, and a liquid crystal display screen is arranged vertically on the inner wall. A camera is arranged in the center of the top of the second compartment.
[0009] The first compartment and the second compartment have a communicating small hole, the size of which matches the size of the mouse, so that the mouse can shuttle between the first compartment and the second compartment.
[0010] The second compartment is provided with a detachable mirror panel at a first height h from the bottom surface, and a fixed platform is provided in the middle of the mirror panel. The fixed platform faces the camera and is used for allowing mice to stand and move freely.
[0011] In one embodiment of the present invention,
[0012] The dimensions of the first compartment and the second compartment are both 403 mm×328 mm×414 mm.
[0013] The inner wall of the first compartment and the inner wall of the second compartment are both made of black acrylic plates.
[0014] The size of the small hole is 40mm×50mm.
[0015] The present invention also provides a monitoring method using the above-mentioned intelligent agent dual-modal visual behavior monitoring system, the monitoring method is used to perform a light-dark shuttle experiment, the detachable mirror panel and the fixed platform are removed from the second compartment, and the monitoring method comprises the following steps:
[0016] S1: Power on the LCD screen to make it appear white, take a picture P1 through the camera, and record the area A1 of the white part in the picture P1;
[0017] S2: placing a black experimental mouse in the first compartment, and setting the initial values of the number of entry and exit times to 0;
[0018] S3: At every time T1, take a picture through the camera and perform binarization processing on it, calculate the difference △A between the area of the white part in the binarized picture and the area A1, and at the same time determine the change trend of △A according to the calculated △A and compare it with the first threshold a1 and the second threshold a2,
[0019] S4: If △A starts from a small value and gradually increases, when △A satisfies △A≥a2, it is considered that the mouse has entered the second compartment from the first compartment, the number of penetrations is increased by 1 and the time t1 at this time is recorded.
[0020] If △A decreases gradually, when △A satisfies △A≤a1, the mouse is considered to have entered the first compartment from the second compartment, the number of exits is increased by 1 and the time t2 is recorded;
[0021] S5: When the time between the first recorded moment t1 and the current moment reaches a preset time T, the light-dark shuttle experiment ends;
[0022] S6: Count the number of times the mouse enters and exits, and count the total time the mouse stays in the second compartment.
[0023] In one embodiment of the present invention, when the mouse is completely located in the second compartment, the camera takes a picture of the mouse at this time and performs binarization processing on it. The area of the black part is A, then a1=A / 3, a2=A / 2.
[0024] In one embodiment of the present invention, a1=1000, a2=1500, and the preset time length T is 5 minutes.
[0025] The present invention also provides a monitoring method using the above-mentioned intelligent dual-modal visual behavior monitoring system. When the monitoring method is used for a visual-motor reaction experiment, the detachable mirror panel and the fixed platform are placed in the second compartment, comprising the following steps:
[0026] S1: placing a black experimental mouse on the fixed platform;
[0027] S2: Control the LCD screen to display a stationary grating for one minute, and then control the LCD screen to display a grating rotation pattern;
[0028] S3: start timing, and within a preset time period, take a picture through the camera every time T2, and perform the following steps S4 to S7 on each picture taken;
[0029] S4: Divide the image into channels and select appropriate thresholds for binarization to detect the outline of the mouse;
[0030] S5: determining the positions of the center of gravity G1 of the mouse body and the center of gravity G2 of the mouse ear contour on the image according to the contours of the mouse body, ear, and tail respectively;
[0031] S6: Outside the mouse ear contour area, select the point on the mouse body contour that is farthest from the body's center of gravity G1 as the location of the nose tip N1, and the distance L1 between N1 and G1 is greater than the distance L2 between N1 and G2;
[0032] S7: On the picture, a first circular area is drawn with the position of the nose tip N1 as the center and the first preset length l1 as the radius, the contour inside the first circular area is taken as the head area of the mouse, the center of gravity of the head area is calculated and taken as the center C1 of the mouse head, a second circular area is drawn with the position of the nose tip N1 of the mouse as the center and the second preset length l2 as the radius, the contour inside the second circular area is taken as the front half area of the mouse, the center of gravity of the front half area is calculated and taken as the center C2 of the mouse head rotation, and the direction from the center C2 of the head rotation to the center C1 of the head is the direction of the mouse head;
[0033] S8: Taking the direction of the mouse head in the first picture as the reference direction and the clockwise direction as the positive direction, obtain the angle values of the directions of the mouse heads in other pictures relative to the reference direction;
[0034] S9: a graph showing the change of the mouse head direction is drawn with time as the horizontal axis and the angle of the mouse head direction relative to the reference direction as the vertical axis;
[0035] S10: Analyze the mouse head direction change map, remove the steeper and more volatile segments, record the curve part that meets the characteristics of the head shaking movement as an optokinetic response, and count the number and time of the mouse's optokinetic response.
[0036] In one embodiment of the present invention, in step S2 , the width, type, rotation speed, rotation direction, brightness, contrast and saturation of the stripes in the grating can be adjusted.
[0037] In one embodiment of the present invention, in step S4, during the binarization process in step S4, the maximum threshold and the minimum threshold of the three channels R, G, and B are both 30 and 255.
[0038] The first preset length l1 and the second preset length l2 are set to one of the following (1) to (3):
[0039] (1) The first preset length l1 is 0.4×L1, and the second preset length l2 is 0.9×L1;
[0040] (2) The first preset length l1 is 30, and the second preset length l2 is 70;
[0041] (3) The center C2 of the mouse's head rotation is located exactly at the connection between the mouse's head and body in the picture.
[0042] In one embodiment of the present invention, in step S10, the curve portion that meets the characteristics of the head shaking action is the curve portion in the mouse head direction change graph whose slope is between 0.1 and 0.65.
[0043] In an embodiment of the present invention, the preset time period is 2 minutes.
[0044] The intelligent dual-modal visual behavior monitoring system provided by the present invention can perform light-dark shuttle experiments and visual-motion reaction experiments to collect experimental mouse movement behavior images and identify and analyze them, use the improved algorithm obtained by deep learning to process the data, output the analysis results, and thus judge the visual function of the mouse. The intelligent dual-modal visual behavior monitoring system provided by the present invention has the advantages of multi-function, low cost, small space, low cost, accurate analysis, reliable results, etc. Researchers can operate independently, and one device can do two experiments, realizing multiple optimizations of human, material and time resources, enhancing operability, and the obtained data is reliable, suitable for wide promotion and application, and can provide strong experimental support for basic scientific research in ophthalmology, and has broad industrialization prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0046] Figure 1 is a schematic diagram of the first compartment and the second compartment;
[0047] Figure 2 The software interface corresponding to the present invention;
[0048] Figure 3 This is a schematic diagram of the extraction results of reference sites in the visual motion response experiment;
[0049] Figure 4 A diagram showing the change in direction of a mouse head according to an embodiment.
[0050] Explanation of the reference numerals: 11 - first compartment; 12 - second compartment; 13 - small hole; 14 - mirror panel; 15 - fixed platform. DETAILED DESCRIPTION
[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0052] The intelligent body dual-modal visual behavior monitoring system provided by the present invention relates to the fields of ophthalmology, image recognition, automatic control, etc., and can be used for light-dark shuttle experiments and visual-motor reaction experiments at the same time.
[0053] In the light-dark shuttle experiment, when the visual function of the mouse is good, the time it stays in the second compartment 12 (bright box) is shorter. The number of times the mouse shuttles between the first compartment 11 and the second compartment 12 within a certain period of time, as well as the time it stays in the second compartment 12, are analyzed to determine the visual function of the mouse.
[0054] In the visual-motion reaction experiment, when the mouse's visual function is good, it will produce a head-shaking reaction to the moving grating, such as following, resetting, and then following again. Based on this principle, the number of times the mouse's behavior occurs during the observation time is analyzed to determine the mouse's visual function.
[0055] Figure 1 is a schematic diagram of the first compartment and the second compartment, as shown Figure 1 As shown, the intelligent dual-modal visual behavior monitoring system provided by the present invention comprises a first compartment 11 and a second compartment 12. The tops of the first compartment 11 and the second compartment 12 are both openable to allow mice to be placed in and taken out for experiments.
[0056] The inner wall of the first compartment 11 is made of an opaque material. The first compartment 11 needs to completely block external light, so the six sides including top, bottom, left, right, front and back are all its inner walls.
[0057] The second compartment 12 has an inner wall and an outer wall, the outer wall is made of an opaque material, and a liquid crystal display screen is arranged on the inner wall in the vertical direction. The second compartment 12 also needs to completely block external light, so the six sides of the upper, lower, left, right, front and back are all its outer walls. When the mouse is placed inside the second compartment 12, generally only its horizontal field of vision needs to be considered, so the inner wall of the second compartment 12 is only in the four directions of front, back, left, right, and there is no need to consider the top and bottom, that is, Figure 1 The second compartment 12 only needs four LCD screens to be arranged in the vertical direction. A camera is arranged in the center of the top of the second compartment 12.
[0058] The first compartment 11 and the second compartment 12 have a communicating small hole 13, the size of which matches the size of the mouse, so that the mouse can shuttle between the first compartment 11 and the second compartment 12.
[0059] The second compartment 12 is provided with a detachable mirror panel 14 at a first height h from the bottom surface. A fixed platform 15 is provided in the middle of the mirror panel 14. The fixed platform 15 faces the camera and is used for the mouse to stand and move freely.
[0060] In this embodiment, the dimensions of the first compartment 11 and the second compartment 12 are both 403 mm × 328 mm × 414 mm, the inner wall of the first compartment 11 and the inner wall of the second compartment 12 are both made of black acrylic plates, and the dimensions of the small hole 13 are 40 mm × 50 mm. Since those skilled in the art can independently set the camera, the inner wall of the first compartment 11, the inner wall and the outer wall of the second compartment 12 according to actual needs, Figure 1 Not shown one by one.
[0061] The present invention also provides a monitoring method using the above-mentioned intelligent dual-modal visual behavior monitoring system. When performing a light-dark shuttle experiment, the detachable mirror panel 14 and the fixed platform 15 are removed from the second compartment 12. The monitoring method comprises the following steps:
[0062] S1: Power on the LCD screen and make it appear white, take a picture P1 through a camera, and record the area A1 of the white part in the picture P1. The LCD screen here refers to the four LCD screens located inside the second compartment 12;
[0063] S2: Place a black experimental mouse in the first compartment, and set the initial values of the number of entry and exit times to 0;
[0064] S3: At every time T1, a picture is taken by the camera and binarized, and the difference △A between the area of the white part in the binarized picture and the area A1 is calculated. At the same time, the change trend of △A is determined based on the calculated △A and compared with the first threshold a1 and the second threshold a2.
[0065] If △A starts from a small value and gradually increases, when △A satisfies △A≥a2, it is considered that the mouse has entered the second compartment 12 from the first compartment 11, the number of penetrations is increased by 1 and the time t1 at this time is recorded.
[0066] If △A decreases gradually, when △A satisfies △A≤a1, it is considered that the mouse has entered the first compartment 11 from the second compartment 12, the number of exits is increased by 1 and the time t2 at this time is recorded;
[0067] S5: When the time between the first recorded moment t1 and the current moment reaches a preset time T, the light-dark shuttle experiment ends;
[0068] S6: Count the number of times the mouse enters and exits, and count the total time the mouse stays in the second compartment 12 .
[0069] In this embodiment, when the mouse is completely located in the second compartment 12, the camera takes a picture of the mouse at this time and performs binarization processing on it. The area of the black part is A, and A is 3000, indicating that the mouse is normally completely located in the second compartment 12. Then a1=A / 3, a2=A / 2, a1=1000, a2=1500, and the preset time length T is 5 minutes.
[0070] The present invention also provides a monitoring method using the above-mentioned intelligent dual-modal visual behavior monitoring system. When the monitoring method is used for a visual-motor reaction experiment, a detachable mirror panel and a fixed platform are placed in the second compartment 12, comprising the following steps:
[0071] S1: Place a black experimental mouse on a fixed platform 15;
[0072] S2: Control the LCD screen to display a stationary grating for one minute, and then control the LCD screen to display a grating rotation pattern. The LCD screen here refers to a total of 4 LCD screens located inside the second compartment 12. At this time, the mirror panel 14 plays the role of extending the grating, enhancing the fear of the mouse so that it is not easy to fall;
[0073] S3: Start timing, and within a preset time period, take a picture through the camera every time T2, and perform the following steps S4 to S7 on each picture taken;
[0074] S4: Divide the image into channels and select appropriate thresholds for binarization to detect the outline of the mouse;
[0075] S5: determining the positions of the center of gravity G1 of the mouse body and the center of gravity G2 of the mouse ear contour on the image according to the contours of the mouse body, ear, and tail respectively;
[0076] S6: Outside the mouse ear contour area, select the point on the mouse body contour that is farthest from the body's center of gravity G1 as the location of the nose tip N1, and the distance L1 between N1 and G1 is greater than the distance L2 between N1 and G2;
[0077] S7: On the picture, a first circular area is drawn with the position of the nose tip N1 as the center and the first preset length l1 as the radius. The contour inside the first circular area is used as the head area of the mouse. The center of gravity of the head area is calculated and used as the center C1 of the mouse head. A second circular area is drawn with the position of the nose tip N1 of the mouse as the center and the second preset length l2 as the radius. The contour inside the second circular area is used as the front half area of the mouse. The center of gravity of the front half area is calculated and used as the center C2 of the mouse head rotation. The direction from the center C2 of the head rotation to the center C1 of the head is the direction of the mouse head. Figure 3The figure shows the extraction result of reference sites in the optokinetic reaction experiment, which is the extraction result of each reference site in one of the pictures;
[0078] S8: Taking the direction of the mouse head in the first picture as the reference direction and the clockwise direction as the positive direction, obtain the angle values of the directions of the mouse heads in other pictures relative to the reference direction;
[0079] S9: With time as the horizontal axis and the angle of the mouse head relative to the reference direction as the vertical axis, a graph of the change in the direction of the mouse head is drawn, such as Figure 4 The figure shows a change in the direction of the head of a mouse according to an embodiment, wherein the ordinate is in radians and the abscissa is in seconds;
[0080] S10: Analyze the mouse head direction change map, remove the steeper and more volatile segments, record the curve part that meets the characteristics of the head shaking movement as an optokinetic response, and count the number and time of the mouse's optokinetic response.
[0081] In this embodiment, in step S2, the width, type, rotation speed, rotation direction, brightness, contrast and saturation of the stripes in the grating can be adjusted. In step S4, during the binarization process in step S4, the maximum threshold and the minimum threshold of the three channels of R, G and B are both 30 and 255, and the first preset length l1 and the second preset length l2 are taken as one of the following (1) to (3):
[0082] (1) The first preset length l1 is 0.4×L1, and the second preset length l2 is 0.9×L1;
[0083] (2) The first preset length l1 is 30, and the second preset length l2 is 70;
[0084] (3) The center C2 of the mouse's head rotation is located exactly at the connection between the mouse's head and body in the picture.
[0085] In this embodiment, in step S10, the curve portion that meets the characteristics of the head shaking action is the curve portion in the mouse head direction change diagram with a slope between 0.1 and 0.65, and the preset time period is 2 minutes.
[0086] When the present invention is actually used, it is necessary to cooperate with a computer and software installed in the computer to carry out. The software works with the camera and the liquid crystal display to control the camera to collect pictures and control the liquid crystal display to display stripes. The software has three main functions: stripe control, video acquisition and processing, and motion state calculation. This embodiment uses MFC as the main control interface to control various parameters in the experiment. Among them, the control program of the stripes uses MFC to design the control interface, and the unity tool is used to draw the stripes. Message communication is used between MFC and unity. Through the control interface of MFC, the width, speed, direction, brightness and other parameters of the stripes can be controlled. Opening the stripe program in the control interface can display the stripes on 4 liquid crystal displays through the connected HDMI line. The programs for image input, data processing, data calculation and data display are directly integrated into the MFC program, and image input, data processing, data calculation and data display can be performed through the control interface of MFC.
[0087] The hardware platform of the present invention has the following structures: the whole system has a power plug, an HDMI interface and a USB interface for connecting to the outside. The power plug is directly connected to the daily power supply interface; the HDMI interface is connected to the HDMI interface of the computer, and the HDMI interface controls the LCD screen through a one-to-four distributor; the USB line is connected to the wide-angle industrial camera and the computer, so as to control the camera to collect images. It needs to be powered by a 220V civilian power supply.
[0088] Figure 2 The software interface corresponding to the present invention is as follows: Figure 2 As shown, the left side is the picture taken by the camera, and the right side is the parameters in the experiment, where:
[0089] Stripe Ctr is the stripe control module. The meanings of various parameters and buttons are as follows:
[0090] Speed: the speed at which the stripes move;
[0091] Bri: brightness of the stripes;
[0092] Size: the width of the stripe;
[0093] Sat: saturation of the stripes;
[0094] Type: the type of stripes (different values correspond to two types: black and white stripes, red and white stripes);
[0095] Con: the contrast between the two colors in the stripe;
[0096] Cyc / deg: The spatial frequency value of the stripes, that is, the number of stripes within a certain length range, which represents the density of the stripes;
[0097] Reverse: By default, the stripes rotate clockwise. If you check this option, the stripes rotate counterclockwise.
[0098] Unityapply: accept stripe parameter modification;
[0099] Unitystart: Start the stripe.
[0100] Exp Ctr is the experimental control module, including Experiment 1 / Ex1 and Experiment 2 / Ex2.
[0101] The parameters in Ex1 include: light-dark shuttle experiment and visual motion reaction experiment
[0102] Radi: The values of the first preset length l1 and the second preset length l2, the default values are 30 and 70;
[0103] Area: in step S4 of the visual motion reaction experiment, when detecting the mouse contour, it is used to limit the minimum and maximum areas of the mouse contour. The default values are 5000 and 20000. In this embodiment, after finding the contour in the image using the findcontours function, the mouse contour is screened and confirmed based on the data. This data is not used for the "light-dark shuttle experiment";
[0104] Slop: In the optokinetic response experiment, in step S10, the numerical range of the slope in the mouse head direction change graph, the default range is 0.1 to 0.65;
[0105] C1, C2: During the visual motion reaction experiment, in step S4, the maximum threshold and minimum threshold of the three channels R, G, and B are both set to 30 and 255 by default;
[0106] The parameters in Ex2 include:
[0107] Bval: binarization threshold in step S4 during the visual motion response experiment, the default value is 50;
[0108] Ain and Aax: the values of a1 and a2, which default to 1000 and 1500 respectively;
[0109] Other parameters:
[0110] FraNum: The total number of mouse images collected, the default value is 20;
[0111] Path: Select the camera. If only one camera is connected, it will be turned on by default. If two or more cameras are connected, the corresponding camera is selected through this parameter. Generally, -1, 0, 1, 2, etc. represent turning on different cameras.
[0112] Mode: The experiment runs in continuous mode; when the mode is selected (input 1), the program automatically runs at four frequencies with Cyc / deg of 0.1, 0.2, 0.3, and 0.4 for 1 minute each, and stops automatically after 4 minutes.
[0113] Title: The type of mouse used in the experiment, entered by the experimenter, records the mouse number and other information.
[0114] Time1 and Time2: The running time of experiments 1 and 2, in seconds, with default values of 60 and 0.
[0115] Apply: Accept parameter changes
[0116] Open: Open the camera
[0117] Pause: Stop the experiment
[0118] Start: Start the experiment
[0119] Result: The result of the experiment:
[0120] Nums: Number of visual-motor responses in Experiment 1
[0121] B2W: Experiment 2, number of penetrations, the number of times the first compartment entered the second compartment;
[0122] W2B: Experiment 2, number of exits, the number of times the person entered the first compartment from the second compartment;
[0123] W-Time: The total time the mouse stayed inside the second compartment.
[0124] It should be noted that the technical means related to size, time, picture acquisition frequency, etc. in the present invention can be adjusted according to the actual experimental situation to meet the actual needs during the experiment.
[0125] The intelligent dual-modal visual behavior monitoring system provided by the present invention can perform light-dark shuttle experiments and visual-motion reaction experiments to collect experimental mouse movement behavior images and identify and analyze them, use the improved algorithm obtained by deep learning to process the data, output the analysis results, and thus judge the visual function of the mouse. The intelligent dual-modal visual behavior monitoring system provided by the present invention has the advantages of multi-function, low cost, small space, low cost, accurate analysis, reliable results, etc. Researchers can operate independently, and one device can do two experiments, realizing multiple optimizations of human, material and time resources, enhancing operability, and the obtained data is reliable, suitable for wide promotion and application, and can provide strong experimental support for basic scientific research in ophthalmology, and has broad industrialization prospects.
[0126] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.
[0127] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.
[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A dual-modal visual behavior monitoring system for intelligent agents, It is characterized in that include: The first compartment and the second compartment, The tops of the first compartment and the second compartment are both openable to allow mice to be placed in and taken out for experiments. The inner wall of the first compartment is made of a light-proof material. The second compartment has an inner wall and an outer wall, the outer wall is made of an opaque material, and a liquid crystal display screen is arranged vertically on the inner wall. A camera is arranged in the center of the top of the second compartment. The first compartment and the second compartment have a communicating small hole, the size of which matches the size of the mouse, so that the mouse can shuttle between the first compartment and the second compartment. The second compartment is provided with a detachable mirror panel at a first height h from the bottom surface, and a fixed platform is provided in the middle of the mirror panel, and the fixed platform faces the camera and is used for the mouse to stand and move freely. When the intelligent agent dual-modal visual behavior monitoring system is used to perform a light-dark shuttle experiment, the detachable mirror panel and the fixed platform are removed from the second compartment, including the following steps: S1: Power on the LCD screen and make it appear white, take a picture P1 with the camera, and record the area A1 of the white part in the picture P1; S2: placing a black experimental mouse in the first compartment, and setting the initial values of the number of entry and exit times to 0; S3: At every time T1, take a picture with the camera and perform binarization processing on it, calculate the difference △A between the area of the white part in the binarized picture and the area A1, and at the same time determine the change trend of △A according to the calculated △A and compare it with the first threshold a1 and the second threshold a2, S4: If △A starts from a small value and gradually increases, when △A satisfies △A≥a2, it is considered that the mouse has entered the second compartment from the first compartment, the number of penetrations is increased by 1 and the time t1 at this time is recorded. If △A decreases gradually, when △A satisfies △A≤a1, the mouse is considered to have entered the first compartment from the second compartment, the number of exits is increased by 1 and the time t2 is recorded; S5: When the time between the first recorded moment t1 and the current moment reaches a preset time T, the light-dark shuttle experiment ends; S6: Count the number of entries and exits, and count the total time the mouse stays in the second compartment. When the dual-modal visual behavior monitoring system for an intelligent agent is used to conduct a visual-motor reaction experiment, the detachable mirror panel and the fixed platform are placed in the second compartment, which includes the following steps: S1′: placing a black experimental mouse on the fixed platform; S2′: Control the liquid crystal display screen to display a stationary grating and keep it for one minute, and then control the liquid crystal display screen to display a grating rotating pattern; S3′: start timing, and within a preset time period, take a picture through the camera every time T2, and perform the following steps S4 to S7 on each picture taken; S4′: Divide the image into channels and select appropriate thresholds for binarization to detect the outline of the mouse; S5′: determining the positions of the center of gravity G1 of the mouse body and the center of gravity G2 of the mouse ear contour on the image according to the contours of the mouse body, ear, and tail respectively; S6′: outside the mouse ear contour area, select the point on the mouse body contour that is farthest from the body's center of gravity G1 as the location of the nose tip N1, and the distance L1 between N1 and G1 is greater than the distance L2 between N1 and G2; S7′: On the picture, a first circular area is drawn with the position of the nose tip N1 as the center and the first preset length l1 as the radius, the contour inside the first circular area is taken as the head area of the mouse, the center of gravity of the head area is calculated and taken as the center C1 of the mouse head, a second circular area is drawn with the position of the nose tip N1 of the mouse as the center and the second preset length l2 as the radius, the contour inside the second circular area is taken as the front half area of the mouse, the center of gravity of the front half area is calculated and taken as the center C2 of the mouse head rotation, and the direction from the center C2 of the head rotation to the center C1 of the head is the direction of the mouse head; S8′: Taking the direction of the mouse head in the first picture as the reference direction and the clockwise direction as the positive direction, obtain the angle values of the directions of the mouse heads in other pictures relative to the reference direction; S9′: a graph showing the change of the mouse head direction, with time as the horizontal axis and the angle of the mouse head direction relative to the reference direction as the vertical axis; S10′: Analyze the mouse head direction change map, remove the steeper and more volatile segments, record the curve part that meets the characteristics of the head shaking movement as an optokinetic response, and count the number and time of the mouse's optokinetic response.
2. The agent dual-modal visual behavior monitoring system according to claim 1, It is characterized in that The dimensions of the first compartment and the second compartment are both 403 mm×328 mm×414 mm. The inner wall of the first compartment and the inner wall of the second compartment are both made of black acrylic plates. The size of the small hole is 40mm×50mm.
3. The monitoring system according to claim 1, It is characterized in that In step S3, when the mouse is completely located in the second compartment, the camera takes a picture of the mouse at this time and performs a binarization process on it. The area of the black part is A, then a1=A / 3, a2=A / 2.
4. The monitoring system according to claim 1, It is characterized in that In step S3, a1=1000, a2=1500, and in step S5, the preset time length T is 5 minutes.
5. The monitoring system according to claim 1, It is characterized in that In step S2 ′, the width, type, rotation speed, rotation direction, brightness, contrast and saturation of the stripes in the grating can be adjusted.
6. The monitoring system according to claim 1, It is characterized in that In step S4′, during the binarization process in step S4′, the maximum threshold and the minimum threshold of the three channels R, G, and B are both 30 and 255. The first preset length l1 and the second preset length l2 are set to one of the following (1) to (3): (1) The first preset length l1 is 0.4×L1, and the second preset length l2 is 0.9×L1; (2) The first preset length l1 is 30, and the second preset length l2 is 70; (3) The center C2 of the mouse's head rotation is located exactly at the connection between the mouse's head and body in the picture.
7. The monitoring system according to claim 1, It is characterized in that In step S10 ′, the curve portion that meets the characteristics of the head shaking action is the curve portion in the mouse head direction change graph whose slope is between 0.1 and 0.
65.
8. The monitoring system according to claim 1, It is characterized in that In step S3', the preset time period is 2 minutes.
Citation Information
Patent Citations
Experimental system for the determination of white rat behavior
CN101584585A
Animal behavior test system and method
CN104382606A
Machine vision-based mouse exploration identification method
CN106611159A
Experimental mouse response monitor
CN107440728A
Two-box type test plant for measuring rat conditional behavior
CN1448110A
