A monitoring method and system for laboratory environment based on multimodality

Abstract

The present application relates to a multimodal laboratory environment monitoring method and system, and relates to the field of environmental monitoring. The method includes obtaining environmental information of each animal cabin in the laboratory during a preset historical time period and a monitoring video of each animal cabin. The environmental information includes a temperature change curve, a humidity change curve, and an oxygen content change curve. The activity change curve of each animal cabin is determined based on the monitoring video. The environmental quality change curve of each animal cabin is determined based on the environmental information. The target cabin whose activity reaches a preset activity threshold is determined based on the activity change curve. The environmental quality change curve and the activity change curve are used to determine whether there is an abnormal cabin in the target cabin. If there is an abnormal cabin, the ventilation device is controlled to ventilate the abnormal cabin. The present application has the effect of facilitating the timely discovery of abnormal cabins and improving the environment of the abnormal cabins.

Description

Technical Field

[0001] The present application relates to the field of environmental monitoring, and in particular to a multimodal laboratory environment monitoring method and system. Background Art

[0002] In biochemistry and other laboratory environments, large numbers of animals, such as mice and rabbits, are required to meet experimental needs. To house these animals, specialized animal holding rooms are typically constructed within the laboratory. These rooms are arranged with multiple small compartments, each housing one or more animals. Because laboratory animals have high environmental requirements and typically need to be maintained at a constant temperature and humidity, the current method for monitoring laboratory animals is to conduct regular manual inspections to check for any abnormalities. However, due to the large number of animals housed, this method results in low monitoring efficiency. When an animal exhibits abnormality, it is difficult to promptly identify the abnormal compartment and implement environmental improvements there. Summary of the Invention

[0003] In order to facilitate the timely discovery of abnormal cabins and improve the environment of abnormal cabins, the present application provides a multi-modal laboratory environment monitoring method and system.

[0004] In a first aspect, the present application provides a multimodal laboratory environment monitoring method, which adopts the following technical solutions:

[0005] A multimodal laboratory environment monitoring method comprising:

[0006] Obtaining environmental information of each animal cabin in the laboratory and surveillance video of each animal cabin during a preset historical time period, wherein the environmental information includes a temperature change curve, a humidity change curve, and an oxygen content change curve;

[0007] Determining an activity change curve for each animal cabin based on the surveillance video;

[0008] determining an environmental quality change curve for each animal cabin based on the environmental information;

[0009] Determining a target cabin whose activity reaches a preset activity threshold based on the activity change curve;

[0010] Determining whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve;

[0011] If there is an abnormal compartment, the ventilation device is controlled to ventilate the abnormal compartment.

[0012] By adopting the above technical solution, environmental information and monitoring video are obtained to facilitate subsequent analysis of whether each animal cabin is abnormal. The monitoring video records the specific activities of the animals in the animal cabin within a preset historical time period. Therefore, the activity change curve of each animal cabin can be accurately determined based on the monitoring video. The environmental information records the environmental changes in the animal cabin within a preset historical time period. Therefore, the environmental quality change curve of each animal cabin can be accurately determined based on the environmental information. The increase in animal activity in the cabin may be caused by a deterioration of the environment or by fighting between animals. Therefore, the target cabin whose activity reaches the preset activity threshold is first determined based on the activity change curve, and then the actual abnormal cabin caused by the deterioration of environmental factors is comprehensively judged based on the environmental quality change curve and the activity change curve of the target cabin. After determining that there is an abnormal cabin, the ventilation device can be controlled to ventilate the abnormal cabin, thereby improving the environment in the abnormal cabin and making the environment in the abnormal cabin reach a level suitable for animal survival. This achieves the effect of facilitating the timely detection of abnormal cabins and improving the environment of the abnormal cabins.

[0013] In another possible implementation, determining the activity change curve of each animal cabin based on the surveillance video includes:

[0014] Splitting the surveillance video of any animal cabin into multiple sub-video segments according to preset intervals;

[0015] Determining leg features, nose tip features, and body center features of each animal in any one of the animal cabins from each sub-video segment;

[0016] Determining a first motion trajectory of the leg feature, a second motion trajectory of the nose tip feature, and a third motion trajectory of the body center feature;

[0017] determining, based on each sub-video segment, the number of times and the frequency at which the nose tip feature of each animal is located on the side wall of any one of the animal compartments;

[0018] determining the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times, and the frequency;

[0019] The activity level of each animal in each sub-video segment is averaged to obtain the sub-activity level corresponding to each sub-video segment;

[0020] An activity change curve of any cabin is generated based on the sub-activity.

[0021] In another possible implementation, determining the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times, and the frequency includes:

[0022] determining a total length of the first motion trajectory and the second motion trajectory;

[0023] determining a product of the total length and a length of the third motion trajectory, wherein the product represents a motion state characteristic value of each animal;

[0024] determining a characteristic value of restlessness for each animal based on the number and frequency;

[0025] The activity level of each animal in the sub-video segments is determined based on the motion state feature value and the anxiety feature value.

[0026] In another possible implementation, determining the environmental quality change curve of each animal cabin based on the environmental information includes:

[0027] dividing the temperature change curve into a plurality of temperature change segments according to preset intervals, dividing the humidity change curve into a plurality of humidity change segments, and dividing the oxygen content change curve into a plurality of oxygen content change segments;

[0028] Ratio calculation of the temperature average value of each temperature change segment and calculation of the absolute value of the first difference between each temperature average value and a preset temperature threshold value;

[0029] Calculating the humidity average value of each humidity change segment and calculating the absolute value of the second difference between each humidity average value and a preset humidity threshold value;

[0030] Calculating an average oxygen content value in each oxygen content variation segment and calculating an absolute value of a third difference between each average oxygen content value and a preset humidity threshold value;

[0031] The absolute value of the first difference, the absolute value of the second difference, and the absolute value of the third difference with the same time interval are summed to obtain the environmental anomaly value of each time interval;

[0032] An environmental quality change curve for each animal cabin was generated based on the environmental outliers at each time interval.

[0033] In another possible implementation, the determining whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve includes:

[0034] An approximate function is obtained by fitting the environmental quality change curve and the activity change curve;

[0035] calculating a first similarity between the approximate function and a preset function, wherein the preset function is determined based on environmental quality change curves and activity change curves of the remaining cabins except the target cabin;

[0036] Determining a plurality of coordinate point groups with the same horizontal coordinate from the environmental quality change curve and the activity change curve, wherein each coordinate point group includes an environmental anomaly value and activity with the same horizontal coordinate;

[0037] Mapping the plurality of coordinate point groups in a preset coordinate system to obtain a scatter plot of the plurality of coordinate point groups, wherein the abscissa of the preset coordinate system is the environmental anomaly value and the ordinate is the activity level;

[0038] Determining the center point and outline of the scatter plot;

[0039] calculating a distance between the center point and a center point of a preset contour map, and a second similarity between the contour and a contour of the preset contour map, wherein the preset contour map is determined based on a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curves and activity change curves of the remaining cabins except the target cabin;

[0040] determining the slope of the approximate function and the area within the contour range of the scatter plot;

[0041] Determining a ratio of the slope to the area, and determining a difference between the ratio and a preset ratio, wherein the preset ratio is a ratio of a slope of a preset function to an area within a contour range of a preset contour graph;

[0042] determining an abnormality score for each target cabin based on the first similarity, the distance, the second similarity, and the difference;

[0043] The target cabin whose abnormal score reaches a preset score threshold is determined as the abnormal cabin.

[0044] In another possible implementation manner, determining the preset function includes:

[0045] An approximate function of each remaining cabin is obtained by fitting the environmental quality change curve and activity change curve of each remaining cabin;

[0046] The preset function is obtained by performing quadratic fitting based on the approximate function of each remaining compartment.

[0047] In another possible implementation, determining the preset profile includes:

[0048] Determine a plurality of coordinate point groups having the same horizontal coordinates in the environmental quality change curve and the activity change curve of each remaining cabin;

[0049] Mapping the multiple coordinate point groups of each remaining cabin into a preset coordinate system to obtain a scatter plot of each remaining cabin;

[0050] Determining the center point of the scatter plot of each remaining compartment, and averaging the center points of the scatter plot of each remaining compartment to obtain the center point of the preset contour map;

[0051] Determine the contour of the scatter plot of each remaining compartment, and perform overlap mapping on the center point of the contour of the scatter plot of each remaining compartment to obtain overlapping contours of overlapping parts of the contours of the scatter plots of all remaining compartments;

[0052] The overlapping contours are connected and completed to obtain the preset contour map.

[0053] In a second aspect, the present application provides a multimodal laboratory environment monitoring system, which adopts the following technical solutions:

[0054] A multimodal laboratory environment monitoring system comprising:

[0055] A data acquisition module is used to obtain environmental information of each animal cabin in the laboratory and surveillance video of each animal cabin during a preset historical period, wherein the environmental information includes a temperature change curve, a humidity change curve, and an oxygen content change curve;

[0056] a first determining module, configured to determine an activity change curve of each animal cabin based on the monitoring video;

[0057] a second determining module, configured to determine an environmental quality change curve of each animal cabin based on the environmental information;

[0058] a third determining module, configured to determine, based on the activity change curve, a target cabin whose activity reaches a preset activity threshold;

[0059] a judgment module, configured to judge whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve;

[0060] The control module is used to control the ventilation device to ventilate the abnormal cabin when there is an abnormal cabin.

[0061] By adopting the above technical solution, the data acquisition module obtains environmental information and surveillance video to facilitate subsequent analysis of whether each animal cabin is abnormal. The surveillance video records the specific activities of the animals in the animal cabin within a preset historical time period. Therefore, the first determination module can accurately determine the activity change curve of each animal cabin based on the surveillance video. The environmental information records the environmental changes in the animal cabin within a preset historical time period. Therefore, the second determination module can accurately determine the environmental quality change curve of each animal cabin based on the environmental information. Increased animal activity in the cabin may be due to a deteriorating environment or due to playfulness between animals. Therefore, the third determination module first determines the target cabin whose activity reaches a preset activity threshold based on the activity change curve. The judgment module then comprehensively determines the actual abnormal cabin caused by deteriorating environmental factors based on the environmental quality change curve and the activity change curve of the target cabin. After determining the existence of an abnormal cabin, the control module controls the ventilation device to ventilate the abnormal cabin, thereby improving the environment in the abnormal cabin to a level suitable for animal survival. This achieves the effect of facilitating the timely detection of abnormal cabins and improving the environment of the abnormal cabin.

[0062] In another possible implementation, when determining the activity change curve of each animal cabin based on the monitoring video, the first determination module is specifically configured to:

[0063] Splitting the surveillance video of any animal cabin into multiple sub-video segments according to preset intervals;

[0064] Determining leg features, nose tip features, and body center features of each animal in any one of the animal cabins from each sub-video segment;

[0065] Determining a first motion trajectory of the leg feature, a second motion trajectory of the nose tip feature, and a third motion trajectory of the body center feature;

[0066] determining, based on each sub-video segment, the number of times and the frequency at which the nose tip feature of each animal is located on the side wall of any one of the animal compartments;

[0067] determining the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times, and the frequency;

[0068] The activity level of each animal in each sub-video segment is averaged to obtain the sub-activity level corresponding to each sub-video segment;

[0069] An activity change curve of any cabin is generated based on the sub-activity.

[0070] In another possible implementation, when the first determination module determines the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times, and the frequency, it is specifically configured to:

[0071] determining a total length of the first motion trajectory and the second motion trajectory;

[0072] determining a product of the total length and a length of the third motion trajectory, wherein the product represents a motion state characteristic value of each animal;

[0073] determining a characteristic value of restlessness for each animal based on the number and frequency;

[0074] The activity level of each animal in the sub-video segments is determined based on the motion state feature value and the anxiety feature value.

[0075] In another possible implementation, when determining the environmental quality change curve of each animal cabin based on the environmental information, the second determination module is specifically configured to:

[0076] dividing the temperature change curve into a plurality of temperature change segments according to preset intervals, dividing the humidity change curve into a plurality of humidity change segments, and dividing the oxygen content change curve into a plurality of oxygen content change segments;

[0077] Ratio calculation of the temperature average value of each temperature change segment and calculation of the absolute value of the first difference between each temperature average value and a preset temperature threshold value;

[0078] Calculating the humidity average value of each humidity change segment and calculating the absolute value of the second difference between each humidity average value and a preset humidity threshold value;

[0079] Calculating an average oxygen content value in each oxygen content variation segment and calculating an absolute value of a third difference between each average oxygen content value and a preset humidity threshold value;

[0080] The absolute value of the first difference, the absolute value of the second difference, and the absolute value of the third difference with the same time interval are summed to obtain the environmental anomaly value of each time interval;

[0081] An environmental quality change curve for each animal cabin was generated based on the environmental outliers at each time interval.

[0082] In another possible implementation, when the judgment module judges whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve, it is specifically configured to:

[0083] An approximate function is obtained by fitting the environmental quality change curve and the activity change curve;

[0084] calculating a first similarity between the approximate function and a preset function, wherein the preset function is determined based on environmental quality change curves and activity change curves of the remaining cabins except the target cabin;

[0085] Determining a plurality of coordinate point groups with the same horizontal coordinate from the environmental quality change curve and the activity change curve, wherein each coordinate point group includes an environmental anomaly value and activity with the same horizontal coordinate;

[0086] Mapping the plurality of coordinate point groups in a preset coordinate system to obtain a scatter plot of the plurality of coordinate point groups, wherein the abscissa of the preset coordinate system is the environmental anomaly value and the ordinate is the activity level;

[0087] Determining the center point and outline of the scatter plot;

[0088] calculating a distance between the center point and a center point of a preset contour map, and a second similarity between the contour and a contour of the preset contour map, wherein the preset contour map is determined based on a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curves and activity change curves of the remaining cabins except the target cabin;

[0089] determining the slope of the approximate function and the area within the contour range of the scatter plot;

[0090] Determining a ratio of the slope to the area, and determining a difference between the ratio and a preset ratio, wherein the preset ratio is a ratio of a slope of a preset function to an area within a contour range of a preset contour graph;

[0091] determining an abnormality score for each target cabin based on the first similarity, the distance, the second similarity, and the difference;

[0092] The target cabin whose abnormal score reaches a preset score threshold is determined as the abnormal cabin.

[0093] In another possible implementation, the system further comprises:

[0094] A first fitting module is used to obtain an approximate function of each remaining cabin by fitting the environmental quality change curve and the activity change curve of each remaining cabin;

[0095] The second fitting module is used to perform quadratic fitting based on the approximate function of each remaining compartment to obtain the preset function.

[0096] In another possible implementation, the system further comprises: determining the preset profile map;

[0097] A fourth determining module is configured to determine a plurality of coordinate point groups having the same horizontal coordinate in the environmental quality change curve and the activity change curve of each remaining cabin;

[0098] A first mapping module is configured to map the plurality of coordinate point groups of each remaining cabin into a preset coordinate system to obtain a scatter plot of each remaining cabin;

[0099] a fifth determining module, configured to determine a center point of the scatter plot of each remaining compartment, and obtain a center point of the preset contour map by averaging the center points of the scatter plot of each remaining compartment;

[0100] A second mapping module is used to determine the contour of the scatter plot of each remaining cabin, and perform coincidence mapping on the center point of the contour of the scatter plot of each remaining cabin to obtain an overlapping contour of the overlapping part of the contours of the scatter plots of all the remaining cabins;

[0101] The completion module is used to complete the overlapping contours to obtain the preset contour map.

[0102] In a third aspect, the present application provides an electronic device, which adopts the following technical solution:

[0103] An electronic device, comprising:

[0104] at least one processor;

[0105] Memory;

[0106] At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one configuration is used to: execute a monitoring method based on a multimodal laboratory environment as shown in any possible implementation of the first aspect.

[0107] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0108] A computer-readable storage medium, when the computer program is executed in a computer, causes the computer to execute the multimodal laboratory environment monitoring method described in any one of the first aspects.

[0109] In summary, this application includes at least one of the following beneficial technical effects:

[0110] Obtaining environmental information and surveillance video facilitates subsequent analysis of whether each animal cabin is abnormal. The surveillance video records the specific activities of animals in the animal cabin within a preset historical time period. Therefore, the activity change curve of each animal cabin can be accurately determined based on the surveillance video. The environmental information records the environmental changes in the animal cabin within a preset historical time period. Therefore, the environmental quality change curve of each animal cabin can be accurately determined based on the environmental information. The increase in animal activity in the cabin may be due to a deterioration in the environment or due to fighting between animals. Therefore, the target cabin whose activity reaches the preset activity threshold is first determined based on the activity change curve, and then the actual abnormal cabin caused by the deterioration of environmental factors is comprehensively judged based on the environmental quality change curve and the activity change curve of the target cabin. After determining that there is an abnormal cabin, the ventilation device can be controlled to ventilate the abnormal cabin, thereby improving the environment in the abnormal cabin and making the environment in the abnormal cabin reach a level suitable for animal survival. This achieves the effect of facilitating the timely discovery of abnormal cabins and improving the environment of the abnormal cabins. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] Figure 1 This is a flow chart of a multimodal laboratory environment monitoring method according to an embodiment of the present application.

[0112] Figure 2 This is a schematic structural diagram of a monitoring system for a multimodal laboratory environment according to an embodiment of the present application.

[0113] Figure 3 It is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0114] The present application is further described in detail below with reference to the accompanying drawings.

[0115] After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.

[0116] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0117] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document, unless otherwise specified, generally indicates an "or" relationship between the related objects.

[0118] The embodiments of the present application are described in further detail below with reference to the accompanying drawings.

[0119] The embodiment of the present application provides a monitoring method for a laboratory environment based on multimodality, which is performed by an electronic device, which can be a server or a terminal device, wherein the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this. The terminal device and the server can be directly or indirectly connected through wired or wireless communication. The embodiment of the present application does not limit this. Figure 1 As shown, the method includes step S101, step S102, step S103, step S104, step S105 and step S106, wherein,

[0120] S101, obtaining environmental information of each animal cabin in the laboratory and surveillance video of each animal cabin in a preset historical time period.

[0121] The environmental information includes temperature change curve, humidity change curve and oxygen content change curve.

[0122] For the embodiment of the present application, the staff can install temperature sensors, humidity sensors and oxygen sensors in each animal cabin in advance to collect the temperature value, humidity value and oxygen content in each animal cabin. The server is connected to the temperature sensor, humidity sensor and oxygen sensor in communication, so that the server can obtain the real-time temperature value, real-time humidity value and real-time oxygen content in each animal cabin, and draw the temperature change curve, humidity change curve and oxygen content change curve based on the obtained data. The electronic device is connected to the server wirelessly, so that the environmental information within the preset historical time period can be obtained. The staff can also install a camera device in each animal cabin in advance, and the electronic device is connected to the camera device through a wire to obtain the surveillance video of the animal cabin collected by each camera device. The preset historical time period can be the past half hour, the past hour, etc., which can be set by the staff through the visual operation interface and stored in the electronic device.

[0123] S102: Determine an activity change curve of each animal cabin based on the surveillance video.

[0124] For the embodiment of the present application, the surveillance video records the specific circumstances of the animal activities in the animal cabin, so the electronic device can accurately determine the activity change curve of each animal cabin within a preset historical time period based on the surveillance video, and facilitate subsequent analysis of whether there is any abnormality in the animal cabin based on the activity change curve.

[0125] S103, determining an environmental quality change curve of each animal cabin based on the environmental information.

[0126] For the embodiment of the present application, after the electronic device obtains the environmental information, the environmental information records the multi-faceted and multi-modal environmental changes in the animal cabin, such as temperature and humidity. Therefore, the electronic device can determine the environmental quality change curve in each animal cabin based on the environmental information. The electronic device facilitates subsequent analysis of whether there is any abnormality in the animal cabin based on the environmental quality change curve.

[0127] S104: Determine a target cabin whose activity reaches a preset activity threshold based on the activity change curve.

[0128] For the embodiment of the present application, when the temperature, humidity and other environmental conditions in the animal cabin make the animals uncomfortable, they will show behaviors such as restlessness and anxiety and want to escape from the animal cabin, that is, the activity level is high at this time. The high activity level of animals in the animal cabin may also be caused by behaviors such as fighting between animals. The preset activity threshold is used as the dividing point whether the activity level is high. After the electronic device determines the activity change curve, the average activity level on the activity change curve can be calculated, and the average value is used to represent the overall level of the activity change curve. The average activity level of an animal cabin reaches the preset activity threshold, indicating that the activity level of the animals in the animal cabin is too high during the preset historical time period. It may be caused by environmental factors in the cabin, but it may also be caused by fighting between animals in the cabin. Therefore, the electronic device first determines the target cabin where the animal activity level is too high.

[0129] S105: Determine whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve.

[0130] For the embodiment of the present application, after the electronic device determines the target cabin, it is necessary to further screen the target cabin to determine the abnormal cabin caused by environmental factors in the cabin. Therefore, the electronic device combines the environmental quality change curve and the activity change curve for comprehensive analysis and judgment to determine the real abnormal cabin caused by environmental factors.

[0131] S106: If there is an abnormal cabin, control the ventilation device to ventilate the abnormal cabin.

[0132] In the embodiment of the present application, the staff can install a ventilation device in each animal cabin. The ventilation device has functions such as air heating, air cooling, and air humidification. After the electronic device determines that there is an abnormal cabin, the electronic device can determine an accurate adjustment strategy based on the environmental information of the abnormal cabin. Specifically, the electronic device can input the environmental information of the abnormal cabin into a trained network model for strategy calculation, thereby determining the ventilation adjustment strategy in the abnormal cabin. The electronic device then controls the ventilation device in the abnormal cabin to ventilate the abnormal cabin according to the determined ventilation adjustment strategy, so that the environment in the abnormal cabin reaches the standard environment for animal survival, achieving the effect of facilitating the timely discovery of abnormal cabins and improving the environment of the abnormal cabins. If there is no abnormal cabin, the electronic device does not perform any operation and performs a new round of monitoring.

[0133] In one possible implementation of the embodiment of the present application, determining the activity change curve of each animal cabin based on the surveillance video in step S102 includes step S1021 (not shown in the figure), step S1022 (not shown in the figure), step S1023 (not shown in the figure), step S1024 (not shown in the figure), step S1025 (not shown in the figure), step S1026 (not shown in the figure), and step S1027 (not shown in the figure), wherein:

[0134] S1021: Segment the surveillance video of any animal cabin into multiple sub-video segments according to preset intervals.

[0135] In this embodiment of the present application, for any animal cabin, the electronic device segments the surveillance video of the cabin into multiple sub-segments. The sub-segments are segmented according to preset intervals, such as three-minute intervals, five-minute intervals, etc. Segmenting the surveillance video into multiple sub-segments facilitates the subsequent determination of a more accurate activity change curve.

[0136] S1022: Determine the leg features, nose tip features, and body center features of each animal in any animal cabin from each sub-video segment.

[0137] In the embodiment of the present application, the electronic device inputs each sub-video into a trained network model for feature recognition, thereby identifying the leg features, nose tip features, and body center features of each animal. Animals move with their legs, so leg features can, to a certain extent, characterize the animal's activity level. The leg features can be any leg of each animal. Animals also explore their surroundings through smell and vision (such as mice, rabbits, etc.). When exploring with head organs such as smell, the animal's head will also move. Therefore, the nose tip features can also, to a certain extent, characterize the animal's activity level. The body center characterizes the animal's position in the cabin, and changes in the animal's position in the cabin can also, to a certain extent, characterize the animal's activity level.

[0138] Specifically, the network model can be a convolutional neural network model, a recurrent neural network model, or other types of network models, which are not limited here.

[0139] S1023 , determining a first motion trajectory of the leg feature, a second motion trajectory of the nose tip feature, and a third motion trajectory of the body center feature.

[0140] In the embodiments of the present application, after the electronic device determines the leg features, it visually tracks the leg features within the surveillance video to obtain a first motion trajectory of each animal's leg features. Similarly, it visually tracks the nose tip features within the surveillance video to obtain a second motion trajectory of each animal's nose tip features, and visually tracks the body center features within the surveillance video to obtain a third motion trajectory of each animal. The longer the first, second, and third motion trajectories, the more frequent the animal's activity and the higher its level of activity.

[0141] S1024 , determining the number and frequency of each animal's nose tip feature being located on the side wall of any animal's cabin based on each sub-video segment.

[0142] For the embodiment of the present application, the electronic device also performs feature recognition on the side walls of the animal cabin, thereby obtaining the edge side wall features of the animal cabin. Specifically, edge detection can be performed on the surveillance video. First, the surveillance video is denoised to obtain a denoised surveillance video, and then the denoised surveillance video is grayscale transformed to obtain a grayscale surveillance video. The position where the grayscale value in the surveillance video steps is the side wall of the animal cabin. The electronic device determines the side wall range of each animal cabin in the surveillance video, and then determines whether the nose tip feature of each animal is within the range of the side wall. If so, the time point when it is within the side wall range is counted and stored. The electronic device can determine the frequency based on the time point each time the nose tip feature is within the side wall range. The higher the frequency and the more times, the more active and anxious the animal is, and it wants to escape from the animal cabin.

[0143] S1025 : Determine the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times, and the frequency.

[0144] For the embodiments of the present application, in summary, the first motion trajectory, the second motion trajectory, the third motion trajectory, and the number and frequency of the nose tip feature located on the side wall of the animal cabin are all key factors affecting the activity of each animal in the sub-video segment. Therefore, the electronic device can comprehensively determine the accurate activity of each animal in each sub-video based on the above five factors.

[0145] S1026 , calculating an average of the activity levels of each animal in each sub-video segment to obtain a sub-activity level corresponding to each sub-video segment.

[0146] For this embodiment of the present application, after the electronic device determines the activity level of each animal in each sub-video segment, it calculates the sub-activity level corresponding to each sub-video segment using the average value calculation formula, that is, the overall activity level of the entire animal cabin during the time period corresponding to the sub-video segment.

[0147] S1027: Generate an activity change curve for any cabin based on the sub-activity.

[0148] In this embodiment of the present application, after determining the sub-activity level for each sub-segment, the electronic device determines the time point corresponding to the sub-segment. For example, the time point corresponding to the midpoint of the sub-segment can be determined as the horizontal coordinate, and the corresponding sub-activity level can be determined as the vertical coordinate, thereby obtaining multiple coordinate points. The electronic device then sequentially connects these multiple coordinate points to obtain an activity level change curve. It should be noted that the electronic device determines the activity level change curve for each animal cabin according to the method of steps S1021 to S1027.

[0149] In a possible implementation of the embodiment of the present application, in step S1025, the activity level of each animal in each sub-video segment is determined based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times, and the frequency, specifically including step Sa (not shown in the figure), step Sb (not shown in the figure), step Sc (not shown in the figure), and step Sd (not shown in the figure), wherein:

[0150] Sa, determine the total length of the first motion trajectory and the second motion trajectory.

[0151] For the embodiment of the present application, the electronic device can determine the number of pixels corresponding to the first motion trajectory and the second motion trajectory, and then use a preset scale to perform a proportional amplification calculation to obtain the length of the first motion trajectory and the length of the second motion trajectory. The preset scale is determined by the staff based on the size of the animal cabin in the monitoring video, so that true and accurate data can be obtained when the length of the first motion trajectory is subsequently calculated. After the electronic device determines the length of the first motion trajectory and the length of the second motion trajectory, the total length can be obtained by summing them. The motion trajectory of the legs and the motion trajectory of the nose tip are both the main activities of animals crawling and sniffing. The longer the motion trajectory of the main activities, the more active the animals are.

[0152] Sb, determines the product of the total length and the length of the third motion trajectory.

[0153] The product represents the characteristic value of the motion state of each animal.

[0154] In this embodiment of the present application, the third motion trajectory represents the positional movement of each animal. The more frequent the positional movement, the more active the animal. Therefore, the electronic device calculates the product of the total length of the first and second motion trajectories and the length of the third motion trajectory. A larger product indicates a more active animal and a higher level of motion state. Therefore, this product represents the motion state of each animal, i.e., the motion state characteristic value.

[0155] Sc, determine the characteristic value of restlessness for each animal based on the number and frequency.

[0156] For the embodiment of the present application, the more times the nose tip feature is located on the side wall of the cabin and the higher the frequency of the nose tip feature contacting the side wall of the cabin, the more the animal wants to escape from the cabin and the more anxious the animal is. Therefore, the staff can set the coefficients corresponding to the number of times and the frequency in advance and store them in the electronic device. After the electronic device determines the number of times and the frequency, it calls the corresponding coefficients for weighted calculation to determine the anxiety characteristic value of each animal in the sub-video segment.

[0157] Sd, determines the activity level of each animal in the sub-video segment based on the motion state feature value and the anxiety feature value.

[0158] In the embodiments of the present application, in summary, both the motion state characteristic value and the anxiety characteristic value are key factors in characterizing the activity level of each animal. Therefore, the electronic device can sum the motion state characteristic value and the anxiety characteristic value to obtain the activity level. Alternatively, the staff can pre-set the coefficients corresponding to the motion state characteristic value and the anxiety characteristic value, and then the electronic device can perform a weighted calculation based on the corresponding coefficients to obtain the activity level. It is more accurate to comprehensively determine the activity level by combining the key characteristic motion trajectory of each animal and the anxiety characteristic value that represents the anxious intention to escape the cabin.

[0159] In one possible implementation of the embodiment of the present application, step S103 determines the environmental quality change curve of each animal cabin based on the environmental information, specifically including step S1031 (not shown in the figure), step S1032 (not shown in the figure), step S1033 (not shown in the figure), step S1034 (not shown in the figure), step S1035 (not shown in the figure), and step S1036 (not shown in the figure), wherein:

[0160] S1031 , dividing the temperature change curve into a plurality of temperature change segments according to preset intervals, dividing the humidity change curve into a plurality of humidity change segments, and dividing the oxygen content change curve into a plurality of oxygen content change segments.

[0161] For the embodiment of the present application, the preset interval is consistent with the preset interval in step S1021. The electronic device divides the temperature change curve, the humidity change curve, and the oxygen content change curve according to the same preset interval to obtain corresponding data segments, namely, multiple temperature change segments, multiple humidity change segments, and multiple oxygen content change segments, so as to facilitate the subsequent analysis of anomalies in combination with the activity change curve to maintain consistency in the time span of the data.

[0162] S1032: Ratio-calculate the average temperature value of each temperature variation segment and calculate the absolute value of the first difference between each average temperature value and a preset temperature threshold.

[0163] In the embodiment of the present application, the electronic device calculates an average temperature value based on the temperature values ​​of each temperature variation segment using an average value calculation formula, uses the average temperature value to represent the overall temperature of each temperature variation segment, and then calculates the absolute value of a first difference between each temperature average value and a preset temperature threshold value. The preset temperature threshold value represents the standard ambient temperature for animal survival, and the absolute value of the first difference represents the degree of deviation of the temperature of each temperature variation segment from the preset temperature threshold value. A larger absolute value of the first difference indicates a greater temperature deviation.

[0164] S1033: Calculate the average humidity value of each humidity variation segment and calculate the absolute value of the second difference between each average humidity value and a preset humidity threshold.

[0165] In the embodiment of the present application, the electronic device calculates an average humidity value based on the humidity value of each humidity variation segment using an average calculation formula. The average humidity value represents the overall humidity of each humidity variation segment. The electronic device then calculates the absolute value of a second difference between each humidity average value and a preset humidity threshold. The preset humidity threshold represents the standard ambient humidity for animal survival. The absolute value of the second difference represents the degree of deviation of the humidity in each humidity variation segment from the preset humidity threshold. A larger absolute value of the second difference indicates a greater humidity deviation.

[0166] S1034: Calculate the average oxygen content value of each oxygen content variation segment and calculate the absolute value of the third difference between each average oxygen content value and the preset humidity threshold.

[0167] In the embodiment of the present application, the electronic device calculates an average oxygen content value based on the oxygen content of each oxygen content variation segment using an average value calculation formula, uses the average oxygen content value to represent the overall oxygen content of each oxygen content variation segment, and then calculates the absolute value of a third difference between each oxygen content average value and a preset oxygen content threshold value. The preset oxygen content threshold value represents the standard environmental oxygen content for animal survival, and the absolute value of the third difference value represents the degree of deviation of the oxygen content of each oxygen content variation segment from the preset oxygen content threshold value. A larger absolute value of the third difference value indicates a greater oxygen content deviation.

[0168] S1035 , summing the absolute value of the first difference, the absolute value of the second difference, and the absolute value of the third difference with the same time interval to obtain an environmental anomaly value for each time interval.

[0169] For the embodiment of the present application, the absolute value of the first difference, the absolute value of the second difference, and the absolute value of the third difference are all key factors representing the quality of the environment. Therefore, the electronic device sums the absolute value of the first difference, the absolute value of the second difference, and the absolute value of the third difference to obtain the environmental anomaly value representing each time interval. The larger the environmental anomaly value, the more abnormal the environment.

[0170] S1036 , generating an environmental quality change curve for each animal cabin based on the environmental abnormality value at each time interval.

[0171] For the embodiment of the present application, after the electronic device determines the sub-environmental anomaly value of each time interval, it determines the time point corresponding to each time interval. For example, the time point corresponding to the middle point of the time interval can be determined as the horizontal coordinate, and the corresponding environmental anomaly value can be determined as the vertical coordinate, thereby obtaining multiple coordinate points. Then, the electronic device can connect the multiple coordinate points in sequence to obtain the living environment quality change curve.

[0172] In a possible implementation of the embodiment of the present application, in step S105, it is determined whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve, which specifically includes steps 1, 2, 3, 4, 5, 6, 7, 8, 9 and 10, wherein:

[0173] Step 1: Fit the environmental quality change curve and the activity change curve to obtain an approximate function.

[0174] In the embodiment of the present application, the electronic device may perform linear fitting based on the coordinate points on the environmental quality change curve to obtain a linear function corresponding to the environmental quality change curve, perform linear fitting based on the activity change curve to obtain a linear function corresponding to the activity change curve, and then average or weighted average the coefficients of the two linear functions to obtain an approximate function. Alternatively, the electronic device may directly map the coordinate points on the environmental quality change curve and the activity change curve into the same coordinate system, and then perform linear fitting on the coordinate points in the coordinate system to obtain an approximate function.

[0175] Step 2: Calculate a first similarity between the approximate function and the preset function.

[0176] The preset function is determined based on the environmental quality change curves and activity change curves of the remaining cabins except the target cabin.

[0177] In this embodiment of the present application, the remaining cabins are all cabins where the animals have normal activity levels, indicating that the environments in the remaining cabins are normal, i.e., the temperature, humidity, and oxygen content all meet the animal survival requirements. The electronic device calculates a first similarity between the approximate function and the preset function. Specifically, this can be achieved by calculating the Euclidean distance or cosine similarity between the approximate function and the preset function. Taking cosine similarity as an example, the closer the cosine similarity is to 1, the greater the similarity, i.e., the more normal the environment in the target cabin is.

[0178] Step three: determine multiple coordinate point groups with the same horizontal coordinate from the environmental quality change curve and the activity change curve, each coordinate point group includes environmental anomaly values ​​and activity with the same horizontal coordinate.

[0179] In the embodiment of the present application, the horizontal coordinates of the environmental quality change curve and the activity change curve are the same, so the electronic device determines a coordinate point group for each environmental anomaly value and activity with the same horizontal coordinate.

[0180] Step 4: Map the multiple coordinate point groups in a preset coordinate system to obtain a scatter plot of the multiple coordinate point groups.

[0181] Among them, the horizontal axis of the preset coordinate system is the environmental anomaly value, and the vertical axis is the activity level.

[0182] In this embodiment of the present application, after the electronic device determines the multiple coordinate point groups corresponding to each target cabin, it maps the multiple coordinate point groups into a preset coordinate system with the horizontal axis representing the environmental anomaly value and the vertical axis representing the activity level, thereby generating a scatter plot. The scatter plot represents the distribution of the relationship between the environmental anomaly value and the activity level within the same time period, or more specifically, the relationship between the environmental anomaly value and the activity level within the same time period.

[0183] Step 5: Determine the center point and outline of the scatter plot.

[0184] In this embodiment of the present application, after the electronic device determines the scatter plot for each target cabin, it averages the horizontal and vertical coordinates of all coordinate points in the scatter plot to obtain the coordinates of the center point. The electronic device then sequentially connects the coordinates of the outermost layer of the scatter plot to obtain the outline of the scatter plot. The center point and outline of the scatter plot are used as the key features of each scatter plot.

[0185] Step six: Calculate the distance between the center point and the center point of the preset contour image, and the second similarity between the contour and the contour of the preset contour image.

[0186] The preset contour map is determined based on a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curves and activity change curves of the remaining cabins except the target cabin.

[0187] For the embodiment of the present application, similarly, the preset contour map is also determined by using multiple coordinate point groups of the remaining cabins. Since the remaining cabins are all cabins with normal animal activity, it means that the environment in the remaining cabins is normal, that is, the temperature, humidity and oxygen content all meet the survival requirements of animals. Therefore, the preset contour map determined based on the remaining cabins represents the relationship between the environmental abnormality value and the activity level, which is normal and meets the survival requirements of organisms. That is, the relationship between the environmental abnormality value and the activity level is consistent with the preset contour map. Figure 1 If the values ​​are consistent or very similar, it indicates that the target cabin's environmental anomalies and activity levels may meet the criteria for animal survival. Electronic devices use the distance formula between two points to calculate the distance between the center point and the center point of the preset contour map. This distance, to a certain extent, represents the difference between the scatter plot and the preset contour map.

[0188] Step seven: determine the slope of the approximate function and the area within the contour range of the scatter plot.

[0189] In the embodiment of the present application, after the electronic device determines the approximate function, it can determine the slope of the approximate function. After the electronic device determines the scatter plot, it can determine the number of complete squares within the outline of the scatter plot, then splice the incomplete squares to obtain the number of spliced ​​squares, and finally sum them to obtain the area within the outline of the scatter plot.

[0190] Step eight, determining the ratio of the slope to the area, and determining the difference between the ratio and a preset ratio.

[0191] The preset ratio is the ratio of the slope of the preset function to the area within the contour range of the preset contour graph.

[0192] For the embodiment of the present application, the electronic device calculates the ratio of the slope to the area, and uses the ratio to characterize the relationship between the approximate function and the scatter plot. Similarly, the electronic device determines in advance the ratio of the slope of the preset function to the area of ​​the preset contour graph, that is, the preset ratio, and then subtracts the ratio of the slope to the area from the preset ratio to obtain the difference, which characterizes the gap between the ratio corresponding to the target cabin and the preset ratio.

[0193] Step nine: determining an abnormality score for each target cabin based on the first similarity, the distance, the second similarity, and the difference.

[0194] In the embodiments of the present application, the first similarity, distance, second similarity, and difference are all key factors in characterizing the degree of abnormality of each target cabin, and their influence varies. Therefore, the staff can pre-set the coefficients corresponding to the first similarity, distance, second similarity, and difference. The electronic device then uses these coefficients to perform a weighted calculation to obtain the abnormality score for each target cabin. To facilitate calculation and ensure the logical consistency of the abnormality score, the electronic device can use the reciprocal of the first similarity and the reciprocal of the second similarity in combination with the distance and difference for weighted calculation.

[0195] Step 10: Determine the target cabin whose abnormality score reaches a preset score threshold as an abnormal cabin.

[0196] In this embodiment of the present application, a preset score threshold serves as the demarcation point for whether an abnormality score is excessively high. After the electronic device determines the abnormality score of each target cabin, it compares it with the preset score threshold. This identifies target cabins whose abnormality scores reach the preset score threshold as abnormal cabins. A comprehensive determination of abnormal cabins using methods such as determining the first similarity, the scatter plot, and the second similarity is more accurate.

[0197] A possible implementation of the embodiment of the present application is to determine the preset function, specifically including step S107 (not shown in the figure) and step S108 (not shown in the figure), wherein:

[0198] S107 , performing fitting based on the environmental quality change curve and activity change curve of each remaining cabin to obtain an approximate function of each remaining cabin.

[0199] S108 , performing quadratic fitting based on the approximate function of each remaining compartment to obtain a preset function.

[0200] For the embodiment of the present application, the electronic device can perform linear fitting based on the coordinate points on the environmental quality change curve of each remaining cabin to obtain a linear function corresponding to the environmental quality change curve, perform linear fitting based on the activity change curve to obtain a linear function about the activity change curve, and then average or weighted average the coefficients of these two linear functions to obtain the approximate function of each remaining cabin. Alternatively, the electronic device directly maps the coordinate points on the environmental quality change curve and the activity change curve of each remaining cabin in the same coordinate system, and then performs linear fitting on the coordinate points in the coordinate system to obtain the approximate function of each remaining cabin. The electronic device then maps all the approximate functions of each remaining cabin to the same coordinate system and performs quadratic fitting to obtain a preset function generated for all the remaining cabins based on all the remaining cabins. The preset function obtained by fitting in this way is more accurate.

[0201] A possible implementation of the embodiment of the present application is to determine a preset contour map, specifically including step S1 (not shown in the figure), step S2 (not shown in the figure), step S3 (not shown in the figure), step S4 (not shown in the figure), and step S5 (not shown in the figure), wherein:

[0202] S1, determining a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curve and the activity change curve of each remaining cabin.

[0203] For the embodiment of the present application, the horizontal coordinates of the environmental quality change curve and the activity change curve of the remaining cabins are the same, so the electronic device determines multiple coordinate point groups for the environmental abnormality values ​​and activity levels with the same horizontal coordinates for each remaining cabin.

[0204] S2, mapping the multiple coordinate point groups of each remaining cabin into a preset coordinate system to obtain a scatter plot of each remaining cabin.

[0205] For an embodiment of the present application, after the electronic device determines multiple coordinate point groups for each remaining cabin, it maps the multiple coordinate point groups for each remaining cabin to a preset coordinate system with the horizontal coordinate being the environmental anomaly value and the vertical coordinate being the activity level, thereby obtaining a scatter plot for each remaining cabin.

[0206] S3, determining the center point of the scatter plot of each remaining compartment, and calculating the average value based on the center point of the scatter plot of each remaining compartment to obtain the center point of the preset contour map.

[0207] For the embodiment of the present application, the electronic device obtains the center point of each scatter plot of each remaining compartment by calculating the average value of all horizontal coordinate points and the average value of vertical coordinate points, and then obtains the center point coordinates of the scatter plots of all remaining compartments by averaging the horizontal and vertical coordinates to obtain the center point coordinates of the preset contour diagram.

[0208] S4, determining the contour of the scatter plot of each remaining compartment, and performing overlap mapping on the center point of the contour of the scatter plot of each remaining compartment to obtain overlapping contours of overlapping parts of the contours of the scatter plots of all remaining compartments.

[0209] For the embodiment of the present application, the electronic device determines the outline of the scatter plot of each remaining cabin in accordance with the method in step five, and then overlaps the center points of all the remaining cabin scatter plots, thereby completing the overlap mapping of the outlines of all the remaining cabin scatter plots. The electronic device can then determine the overlapping part of the outlines of all the remaining cabin scatter plots, that is, the overlapping outline.

[0210] S5, completing the overlapping contours by connecting lines to obtain a preset contour map.

[0211] For the embodiment of the present application, after the electronic device determines the overlapping contours, it connects the disconnected areas according to the endpoints of the adjacent overlapping contours to obtain a preset contour map. The preset contour map is determined based on the contours of the scatter plots of all the remaining cabins, thereby making the preset contour map more accurate and better able to represent the relationship between environmental anomalies and activity levels that meet the animal survival requirements.

[0212] The above embodiment introduces a monitoring method based on a multimodal laboratory environment from the perspective of method flow. The following embodiment introduces a monitoring system based on a multimodal laboratory environment from the perspective of a virtual module or virtual unit. For details, please refer to the following embodiment.

[0213] The embodiment of the present application provides a monitoring system 20 based on a multimodal laboratory environment, such as Figure 2 As shown, the multimodal laboratory environment monitoring system 20 may specifically include:

[0214] The data acquisition module 201 is used to obtain environmental information of each animal cabin in the laboratory and the monitoring video of each animal cabin during a preset historical period. The environmental information includes a temperature change curve, a humidity change curve, and an oxygen content change curve;

[0215] A first determination module 202 is configured to determine an activity change curve of each animal cabin based on the surveillance video;

[0216] A second determining module 203 is configured to determine an environmental quality change curve for each animal cabin based on the environmental information;

[0217] The third determining module 204 is configured to determine a target cabin whose activity reaches a preset activity threshold based on the activity change curve;

[0218] A judgment module 205 is configured to judge whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve;

[0219] The control module 206 is used to control the ventilation device to ventilate the abnormal cabin when there is an abnormal cabin.

[0220] The embodiment of the present application discloses a monitoring system 20 based on a multimodal laboratory environment, wherein a data acquisition module 201 acquires environmental information and monitoring video to facilitate subsequent analysis of whether each animal cabin is abnormal. The monitoring video records the specific activities of the animals in the animal cabin within a preset historical time period. Therefore, the first determination module 202 can accurately determine the activity change curve of each animal cabin based on the monitoring video. The environmental information records the environmental changes of the animal cabin within a preset historical time period. Therefore, the second determination module 203 can accurately determine the environmental quality change curve of each animal cabin based on the environmental information. The increase in animal activity in the cabin may be due to It may be caused by environmental deterioration or by fighting between animals. Therefore, the third determination module 204 first determines the target cabin whose activity reaches the preset activity threshold based on the activity change curve. The judgment module 205 then comprehensively judges the real abnormal cabin caused by the deterioration of environmental factors based on the environmental quality change curve and the activity change curve of the target cabin. After determining that there is an abnormal cabin, the control module 206 controls the ventilation device to ventilate the abnormal cabin, thereby improving the environment in the abnormal cabin and making the environment in the abnormal cabin reach a level suitable for animal survival, thereby achieving the effect of facilitating timely detection of abnormal cabins and improving the environment of the abnormal cabins.

[0221] In one possible implementation of the embodiment of the present application, when determining the activity change curve of each animal cabin based on the surveillance video, the first determination module 202 is specifically configured to:

[0222] Splitting the surveillance video of any animal cabin into multiple sub-video segments according to preset intervals;

[0223] Determine the leg features, nose tip features, and body center features of each animal in any animal cabin from each sub-video segment;

[0224] Determine a first motion trajectory of the leg feature, a second motion trajectory of the nose tip feature, and a third motion trajectory of the body center feature;

[0225] determining, based on each sub-video segment, the number of times and the frequency with which the nose tip feature of each animal was located on the side wall of any of the animal's compartments;

[0226] determining the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times, and the frequency;

[0227] The activity level of each animal in each sub-video segment is averaged to obtain the sub-activity level corresponding to each sub-video segment;

[0228] Generate an activity change curve for any cabin based on the sub-activity.

[0229] In one possible implementation of the embodiment of the present application, when the first determination module 202 determines the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times, and the frequency, it is specifically configured to:

[0230] determining a total length of the first motion trajectory and the second motion trajectory;

[0231] determining a product of the total length and the length of the third motion trajectory, wherein the product represents a characteristic value of the motion state of each animal;

[0232] Determine the characteristic value of agitation for each animal based on the number and frequency;

[0233] The activity level of each animal in each sub-video segment is determined based on the motion state feature value and the anxiety feature value.

[0234] In one possible implementation of the embodiment of the present application, when determining the environmental quality change curve of each animal cabin based on the environmental information, the second determining module 203 is specifically configured to:

[0235] Dividing the temperature change curve into a plurality of temperature change segments according to preset intervals, dividing the humidity change curve into a plurality of humidity change segments, and dividing the oxygen content change curve into a plurality of oxygen content change segments;

[0236] Ratio calculation of the temperature average value of each temperature change segment and calculation of the absolute value of the first difference between each temperature average value and a preset temperature threshold value;

[0237] Calculating the humidity average value of each humidity change segment and calculating the absolute value of the second difference between each humidity average value and a preset humidity threshold value;

[0238] Calculating an average oxygen content value in each oxygen content variation segment and calculating an absolute value of a third difference between each average oxygen content value and a preset humidity threshold value;

[0239] The absolute value of the first difference, the absolute value of the second difference, and the absolute value of the third difference with the same time interval are summed to obtain the environmental anomaly value of each time interval;

[0240] An environmental quality change curve for each animal cabin was generated based on the environmental outliers at each time interval.

[0241] In one possible implementation of the embodiment of the present application, when the judgment module 205 judges whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve, it is specifically configured to:

[0242] An approximate function is obtained by fitting the environmental quality change curve and the activity change curve;

[0243] calculating a first similarity between the approximate function and a preset function, where the preset function is determined based on environmental quality change curves and activity change curves of the remaining cabins except the target cabin;

[0244] Determine multiple coordinate point groups with the same horizontal coordinate from the environmental quality change curve and the activity change curve, each coordinate point group includes environmental anomaly values ​​and activity with the same horizontal coordinate;

[0245] Mapping the multiple coordinate point groups in a preset coordinate system to obtain a scatter plot of the multiple coordinate point groups, where the horizontal axis of the preset coordinate system is the environmental outlier value and the vertical axis is the activity level;

[0246] Determine the center point and outline of the scatter plot;

[0247] Calculating a distance between the center point and a center point of a preset contour map, and a second similarity between the contour and the contour of the preset contour map, wherein the preset contour map is determined based on a plurality of coordinate point groups having the same horizontal coordinate in the environmental quality change curves and activity change curves of the remaining cabins except the target cabin;

[0248] Determine the slope of the approximate function and the area within the contour of the scatter plot;

[0249] Determining a ratio of the slope to the area, and determining a difference between the ratio and a preset ratio, wherein the preset ratio is a ratio of the slope of the preset function to the area within the contour range of the preset contour graph;

[0250] determining an anomaly score for each target cabin based on the first similarity, the distance, the second similarity, and the difference;

[0251] The target cabin whose abnormal score reaches the preset score threshold is determined as an abnormal cabin.

[0252] In a possible implementation of the embodiment of the present application, the system 20 determines the preset function and further includes:

[0253] A first fitting module is used to obtain an approximate function of each remaining cabin by fitting the environmental quality change curve and the activity change curve of each remaining cabin;

[0254] The second fitting module is used to perform quadratic fitting based on the approximate function of each remaining compartment to obtain a preset function.

[0255] In a possible implementation of the embodiment of the present application, the system 20 determines a preset profile map and further includes:

[0256] A fourth determining module is configured to determine a plurality of coordinate point groups having the same horizontal coordinate in the environmental quality change curve and the activity change curve of each remaining cabin;

[0257] A first mapping module is configured to map the plurality of coordinate point groups of each remaining cabin into a preset coordinate system to obtain a scatter plot of each remaining cabin;

[0258] a fifth determining module, configured to determine a center point of the scatter plot of each remaining compartment, and obtain a center point of a preset contour map by averaging the center points of the scatter plot of each remaining compartment;

[0259] A second mapping module is used to determine the contour of the scatter plot of each remaining cabin, and perform coincidence mapping on the center point of the contour of the scatter plot of each remaining cabin to obtain an overlapping contour of the overlapping part of the contours of the scatter plots of all the remaining cabins;

[0260] The completion module is used to connect and complete overlapping contours to obtain a preset contour map.

[0261] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the multimodal laboratory environment monitoring system 20 described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0262] An electronic device is provided in an embodiment of the present application, such as Figure 3 As shown, Figure 3 The electronic device 30 shown includes a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302. Optionally, the electronic device 30 may further include a transceiver 304. It should be noted that in actual applications, the number of transceivers 304 is not limited to one, and the structure of the electronic device 30 does not constitute a limitation on the embodiments of the present application.

[0263] Processor 301 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0264] Bus 302 may include a path for transmitting information between the above components. Bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 3 Only one thick line is used in the diagram, but it does not mean that there is only one bus or one type of bus.

[0265] The memory 303 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0266] The memory 303 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the content shown in the above method embodiment.

[0267] Electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. They may also include servers, etc. Figure 3 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0268] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding contents of the aforementioned method embodiment. Compared with the related art, the acquisition of environmental information and monitoring video in the embodiment of the present application facilitates the subsequent analysis of whether each animal cabin is abnormal. The monitoring video records the specific activities of the animals in the animal cabin within a preset historical time period. Therefore, the activity change curve of each animal cabin can be accurately determined based on the monitoring video. The environmental information records the environmental changes in the animal cabin within a preset historical time period. Therefore, the environmental quality change curve of each animal cabin can be accurately determined based on the environmental information. The increase in animal activity in the cabin may be caused by environmental deterioration or by fighting between animals. Therefore, the target cabin whose activity reaches the preset activity threshold is first determined based on the activity change curve, and then the real abnormal cabin caused by the deterioration of environmental factors is comprehensively judged based on the environmental quality change curve and the activity change curve of the target cabin. After determining that there is an abnormal cabin, the ventilation device can be controlled to ventilate the abnormal cabin, thereby improving the environment in the abnormal cabin and making the environment in the abnormal cabin reach a level suitable for animal survival. This achieves the effect of facilitating the timely detection of abnormal cabins and improving the environment of the abnormal cabins.

[0269] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0270] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A multimodal laboratory environment monitoring method, characterized in that: include: Obtaining environmental information and surveillance video of each animal cabin in the laboratory during a preset historical time period, the environmental information including a temperature change curve, a humidity change curve, and an oxygen content change curve; and determining an activity change curve for each animal cabin based on the surveillance video; Determine an environmental quality change curve for each animal cabin based on the environmental information; determine a target cabin whose activity reaches a preset activity threshold based on the activity change curve; and determine whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve; If there is an abnormal compartment, controlling the ventilation device to ventilate the abnormal compartment; The method of determining the activity change curve of each animal cabin based on the surveillance video includes: dividing the surveillance video of any animal cabin into a plurality of sub-video segments according to preset intervals; determining the leg features, nose tip features, and body center features of each animal in the any animal cabin from each sub-video segment; determining a first motion trajectory of the leg features, a second motion trajectory of the nose tip features, and a third motion trajectory of the body center features; determining the number and frequency of each animal's nose tip feature being located on the side wall of the any animal cabin based on each sub-video segment; and determining the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number and frequency. The activity level of each animal in each sub-video segment is averaged to obtain the sub-activity level corresponding to each sub-video segment; an activity level change curve of any animal cabin is generated based on the sub-activity level; the activity level of each animal in each sub-video segment is determined based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times and the frequency, including: determining the total length of the first motion trajectory and the second motion trajectory; determining the product of the total length and the length of the third motion trajectory, the product representing the motion state characteristic value of each animal; determining the anxiety characteristic value of each animal based on the number of times and the frequency; and determining the activity level of each animal in each sub-video segment based on the motion state characteristic value and the anxiety characteristic value.

2. The method for monitoring a laboratory environment based on multimodality according to claim 1, characterized in that: The method for determining the environmental quality change curve of each animal cabin based on the environmental information includes: dividing the temperature change curve into multiple temperature change segments according to preset intervals, dividing the humidity change curve into multiple humidity change segments, and dividing the oxygen content change curve into multiple oxygen content change segments; calculating the temperature average value of each temperature change segment by ratio and calculating the absolute value of the first difference between each temperature average value and the preset temperature threshold; calculating the humidity average value of each humidity change segment and calculating the absolute value of the second difference between each humidity average value and the preset humidity threshold; calculating the oxygen content average value of each oxygen content change segment and calculating the absolute value of the third difference between each oxygen content average value and the preset humidity threshold; summing the absolute value of the first difference, the absolute value of the second difference, and the absolute value of the third difference with the same time interval to obtain the environmental abnormality value of each time interval; generating the environmental quality change curve of each animal cabin based on the environmental abnormality value of each time interval.

3. The method for monitoring a laboratory environment based on multimodality according to claim 1, characterized in that: The method of judging whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve includes: obtaining an approximate function by fitting the environmental quality change curve and the activity change curve; calculating a first similarity between the approximate function and a preset function, wherein the preset function is determined based on the environmental quality change curve and the activity change curve of the remaining cabins except the target cabin; determining a plurality of coordinate point groups with the same horizontal coordinate from the environmental quality change curve and the activity change curve, each coordinate point group including an environmental abnormality value and an activity with the same horizontal coordinate; mapping the plurality of coordinate point groups in a preset coordinate system to obtain a scatter plot of the plurality of coordinate point groups, wherein the horizontal coordinate of the preset coordinate system is the environmental abnormality value and the vertical coordinate is the activity; determining the scatter plot. the center point and outline of the scatter plot; calculating the distance between the center point and the center point of the preset outline graph, and the second similarity between the outline and the outline of the preset outline graph, wherein the preset outline graph is determined based on a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curves and activity change curves of the remaining cabins except the target cabin; determining the slope of the approximate function and the area within the outline range of the scatter plot; determining the ratio of the slope to the area, and determining the difference between the ratio and a preset ratio, wherein the preset ratio is the ratio of the slope of the preset function to the area within the outline range of the preset outline graph; determining the abnormality score of each target cabin based on the first similarity, the distance, the second similarity and the difference; determining the target cabin whose abnormality score reaches the preset score threshold as the abnormal cabin.

4. The method for monitoring a laboratory environment based on multimodality according to claim 3, characterized in that: Determining the preset function includes: performing fitting based on the environmental quality change curve and the activity change curve of each remaining cabin to obtain an approximate function of each remaining cabin; and performing quadratic fitting based on the approximate function of each remaining cabin to obtain the preset function.

5. The method for monitoring a laboratory environment based on multimodality according to claim 3, characterized in that: Determining the preset contour map includes: determining multiple coordinate point groups with the same horizontal coordinates in the environmental quality change curve and the activity change curve of each remaining cabin; mapping the multiple coordinate point groups of each remaining cabin to the preset coordinate system to obtain a scatter plot of each remaining cabin; determining the center point of the scatter plot of each remaining cabin, and averaging the center points of the scatter plots of each remaining cabin to obtain the center point of the preset contour map; determining the contour of the scatter plot of each remaining cabin, and overlappingly mapping the center points of the contours of the scatter plots of each remaining cabin to obtain overlapping contours of the overlapping parts of the contours of the scatter plots of all the remaining cabins; and connecting and completing the overlapping contours to obtain the preset contour map.

6. A multimodal laboratory environment monitoring system, characterized in that: include: A data acquisition module is used to obtain environmental information of each animal cabin in the laboratory and surveillance video of each animal cabin during a preset historical period, wherein the environmental information includes a temperature change curve, a humidity change curve, and an oxygen content change curve; a first determining module, configured to determine an activity change curve of each animal cabin based on the monitoring video; The method for determining the activity change curve of each animal cabin based on the surveillance video includes: dividing the surveillance video of any animal cabin into multiple sub-video segments according to preset intervals; determining the leg features, nose tip features and body center features of each animal in the any animal cabin from each sub-video segment; determining a first motion trajectory of the leg features, a second motion trajectory of the nose tip features and a third motion trajectory of the body center features; determining the number and frequency of each animal's nose tip features being located on the side wall of the any animal cabin based on each sub-video segment; and determining the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number and the frequency. ; Calculate the average of the activity levels of each animal in each sub-video segment to obtain the sub-activity level corresponding to each sub-video segment; generate an activity level change curve for any animal cabin based on the sub-activity level; the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times and the frequency includes: determining the total length of the first motion trajectory and the second motion trajectory; determining the product of the total length and the length of the third motion trajectory, the product representing the motion state characteristic value of each animal; determining the anxiety characteristic value of each animal based on the number of times and the frequency; determining the activity level of each animal in each sub-video segment based on the motion state characteristic value and the anxiety characteristic value; a second determining module, configured to determine an environmental quality change curve of each animal cabin based on the environmental information; a third determining module, configured to determine, based on the activity change curve, a target cabin whose activity reaches a preset activity threshold; a judgment module, configured to judge whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve; The control module is used to control the ventilation device to ventilate the abnormal cabin when there is an abnormal cabin.

7. An electronic device, characterized in that: It includes: at least one processor; Memory; At least one application, wherein the at least one application is stored in the memory and is configured to be executed by the at least one processor, the at least one application being used to execute the multimodal laboratory environment monitoring method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed in a computer, the computer is caused to execute the multimodal laboratory environment monitoring method according to any one of claims 1 to 5.

Images