Laboratory environment monitoring method and system based on multiple modes

Abstract

The invention relates to a multi-mode-based laboratory environment monitoring method and system, and relates to the field of environment monitoring, and the method comprises the steps: obtaining environment information of each animal cabin in a laboratory in a preset historical time period and a monitoring video of each animal cabin, the environment information comprises a temperature change curve, a humidity change curve and an oxygen content change curve, determining an activeness change curve of each animal cabin according to the monitoring video, determining an environment quality change curve of each animal cabin according to the environment information, determining a target cabin of which the activeness reaches a preset activeness threshold according to the activeness change curve, and determining a target cabin of which the activeness reaches a preset activeness threshold according to the target cabin. And judging whether an abnormal cabin exists in the target cabin or not according to the environment quality change curve and the activity change curve, and if yes, controlling a ventilation device to ventilate the abnormal cabin. The method has the effects that the abnormal cabin can be conveniently found in time, and the environment of the abnormal cabin is improved.

Description

Technical Field

[0001] The present application relates to the field of environmental monitoring, and in particular to a multi-modal laboratory environment monitoring method and system. Background Art

[0002] In laboratory environments such as biochemistry, a large number of animals such as mice and rabbits are needed to meet experimental needs. In order to house these animals, a special animal placement room is usually built in the laboratory. There are multiple small cabins arranged in the animal placement room, and one or more animals are placed in each cabin. Since animals in the laboratory have high requirements for the environment, they usually need to be kept in a constant environment such as specific temperature and humidity. The current monitoring method for animals in the laboratory is still to conduct regular manual inspections to check whether the animals are abnormal. However, due to the large number of animals placed, this method leads to low monitoring efficiency. When an animal is abnormal, it is not easy to promptly discover the abnormal cabin and improve the environment of the abnormal cabin. Summary of the invention

[0003] In order to facilitate timely discovery of abnormal cabins and improve the environment of the abnormal cabins, the present application provides a monitoring method and system for laboratory environments based on multimodality.

[0004] In a first aspect, the present application provides a monitoring method for a laboratory environment based on multimodality, which adopts the following technical solution: A multi-modal laboratory environment monitoring method, comprising: Obtaining environmental information of each animal cabin in the laboratory during a preset historical time period and surveillance video of each animal cabin, wherein the environmental information includes a temperature change curve, a humidity change curve, and an oxygen content change curve; Determine an activity change curve of each animal cabin based on the monitoring video; Determine an environmental quality change curve for each animal cabin based on the environmental information; Determining a target cabin whose activity reaches a preset activity threshold based on the activity change curve; Determining whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve; If there is an abnormal compartment, the ventilation device is controlled to ventilate the abnormal compartment.

[0005] By adopting the above technical scheme, environmental information and monitoring video are obtained to facilitate subsequent analysis of whether each animal cabin is abnormal. The monitoring video records the specific activities of the animals in the animal cabin within a preset historical time period. Therefore, the activity change curve of each animal cabin can be accurately determined based on the monitoring video. The environmental information records the environmental changes in the animal cabin within a preset historical time period. Therefore, the environmental quality change curve of each animal cabin can be accurately determined based on the environmental information. The increase in the activity of animals in the cabin may be caused by the deterioration of the environment or by the animals fighting with each other. Therefore, the target cabin whose activity reaches the preset activity threshold is first determined based on the activity change curve, and then the real abnormal cabin caused by the deterioration of environmental factors is comprehensively judged based on the environmental quality change curve and the activity change curve of the target cabin. After determining that there is an abnormal cabin, the ventilation device can be controlled to ventilate the abnormal cabin, thereby improving the environment in the abnormal cabin and making the environment in the abnormal cabin reach a level suitable for animal survival, thereby achieving the effect of facilitating timely discovery of abnormal cabins and improving the environment of the abnormal cabins.

[0006] In another possible implementation, determining the activity change curve of each animal cabin based on the monitoring video includes: Splitting the surveillance video of any animal cabin into a plurality of sub-video segments according to preset intervals; Determine the leg features, nose tip features and body center features of each animal in any one of the animal cabins from each sub-video segment; Determine a first motion trajectory of the leg feature, a second motion trajectory of the nose tip feature, and a third motion trajectory of the body center point feature; Determining the number of times and the frequency at which the nose tip feature of each animal is located on the side wall of any of the animal compartments based on each of the sub-video segments; Determine the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times and the frequency; The activity level of each animal in each sub-video segment is averaged to obtain the sub-activity level corresponding to each sub-video segment; An activity change curve of any cabin is generated based on the sub-activity.

[0007] In another possible implementation, determining the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times and the frequency includes: Determining the total length of the first motion trajectory and the second motion trajectory; Determine the product of the total length and the length of the third motion trajectory, wherein the product represents a motion state characteristic value of each animal; determining a characteristic value of agitation for each animal based on the number and frequency; The activity level of each animal in each sub-video segment is determined based on the motion state feature value and the anxiety feature value.

[0008] In another possible implementation, determining the environmental quality change curve of each animal cabin based on the environmental information includes: Dividing the temperature change curve into a plurality of temperature change segments according to preset intervals, dividing the humidity change curve into a plurality of humidity change segments, and dividing the oxygen content change curve into a plurality of oxygen content change segments; The ratio calculates the temperature average value of each temperature change section and calculates the absolute value of the first difference between each temperature average value and a preset temperature threshold value; Calculating the humidity average value of each humidity variation segment and calculating the absolute value of the second difference between each humidity average value and a preset humidity threshold value; Calculating an average oxygen content value in each oxygen content variation segment and calculating an absolute value of a third difference between each average oxygen content value and a preset humidity threshold value; The absolute value of the first difference, the absolute value of the second difference and the absolute value of the third difference with the same time interval are summed to obtain the environmental anomaly value of each time interval; An environmental quality change curve for each animal cabin is generated based on the environmental outliers at each time interval.

[0009] In another possible implementation, judging whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve includes: An approximate function is obtained by fitting based on the environmental quality change curve and the activity change curve; Calculating a first similarity between the approximate function and a preset function, wherein the preset function is determined according to environmental quality change curves and activity change curves of the remaining cabins except the target cabin; Determine a plurality of coordinate point groups with the same horizontal coordinate from the environmental quality change curve and the activity change curve, each coordinate point group including an environmental anomaly value and activity with the same horizontal coordinate; Mapping the plurality of coordinate point groups in a preset coordinate system to obtain a scatter plot of the plurality of coordinate point groups, wherein the abscissa of the preset coordinate system is the environmental anomaly value, and the ordinate is the activity degree; Determine the center point and the outline of the scatter plot; Calculating a distance between the center point and a center point of a preset contour map, and a second similarity between the contour and a contour of the preset contour map, wherein the preset contour map is determined based on a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curves and activity change curves of the remaining cabins except the target cabin; Determining the slope of the approximate function and the area within the contour range of the scatter plot; Determine a ratio of the slope to the area, and determine a difference between the ratio and a preset ratio, wherein the preset ratio is a ratio of a slope of a preset function to an area within a contour range of a preset contour graph; Determining an abnormality score for each target cabin based on the first similarity, the distance, the second similarity, and the difference; The target cabin whose abnormal score reaches a preset score threshold is determined as the abnormal cabin.

[0010] In another possible implementation manner, determining the preset function includes: An approximate function of each remaining cabin is obtained by fitting the environmental quality change curve and activity change curve of each remaining cabin; The preset function is obtained by performing quadratic fitting based on the approximate function of each remaining compartment.

[0011] In another possible implementation manner, determining the preset profile map includes: Determine a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curve and the activity change curve of each remaining cabin; Mapping multiple coordinate point groups of each remaining cabin into a preset coordinate system to obtain a scatter plot of each remaining cabin; Determine the center point of the scatter plot of each remaining compartment, and obtain the center point of the preset contour map by averaging the center points of the scatter plot of each remaining compartment; Determine the contour of the scatter plot of each remaining cabin, and perform coincidence mapping on the center point of the contour of the scatter plot of each remaining cabin to obtain the overlapping contour of the overlapping part of the contours of the scatter plots of all the remaining cabins; The overlapping contours are connected and completed to obtain the preset contour map.

[0012] In a second aspect, the present application provides a monitoring system for a laboratory environment based on a multimodal system, which adopts the following technical solution: A monitoring system for a laboratory environment based on multimodality, comprising: A data acquisition module, used to acquire environmental information of each animal cabin in the laboratory during a preset historical period and a monitoring video of each animal cabin, wherein the environmental information includes a temperature change curve, a humidity change curve, and an oxygen content change curve; A first determination module, configured to determine an activity change curve of each animal cabin based on the monitoring video; A second determination module is used to determine an environmental quality change curve of each animal cabin based on the environmental information; A third determination module is used to determine a target cabin whose activity reaches a preset activity threshold based on the activity change curve; A judgment module, used for judging whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve; The control module is used to control the ventilation device to ventilate the abnormal cabin when there is an abnormal cabin.

[0013] By adopting the above technical solution, the data acquisition module obtains environmental information and monitoring video to facilitate the subsequent analysis of whether each animal cabin is abnormal. The monitoring video records the specific activities of the animals in the animal cabin within the preset historical time period. Therefore, the first determination module can accurately determine the activity change curve of each animal cabin according to the monitoring video. The environmental information records the environmental changes of the animal cabin within the preset historical time period. Therefore, the second determination module can accurately determine the environmental quality change curve of each animal cabin according to the environmental information. The increase in the activity of animals in the cabin may be caused by the deterioration of the environment or the fight between animals. Therefore, the third determination module first determines the target cabin whose activity reaches the preset activity threshold according to the activity change curve, and the judgment module then comprehensively judges the real abnormal cabin caused by the deterioration of environmental factors according to the environmental quality change curve and the activity change curve of the target cabin. After determining that there is an abnormal cabin, the control module controls the ventilation device to ventilate the abnormal cabin, thereby improving the environment in the abnormal cabin and making the environment in the abnormal cabin reach a level suitable for animal survival, thereby achieving the effect of facilitating timely discovery of abnormal cabins and improving the environment of the abnormal cabins.

[0014] In another possible implementation, when the first determination module determines the activity change curve of each animal cabin based on the monitoring video, it is specifically used to: Splitting the surveillance video of any animal cabin into a plurality of sub-video segments according to preset intervals; Determine the leg features, nose tip features and body center features of each animal in any one of the animal cabins from each sub-video segment; Determine a first motion trajectory of the leg feature, a second motion trajectory of the nose tip feature, and a third motion trajectory of the body center point feature; Determining the number of times and the frequency at which the nose tip feature of each animal is located on the side wall of any of the animal compartments based on each of the sub-video segments; Determine the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times and the frequency; The activity level of each animal in each sub-video segment is averaged to obtain the sub-activity level corresponding to each sub-video segment; An activity change curve of any cabin is generated based on the sub-activity.

[0015] In another possible implementation, when the first determination module determines the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times and the frequency, it is specifically configured to: Determining the total length of the first motion trajectory and the second motion trajectory; Determine the product of the total length and the length of the third motion trajectory, wherein the product represents a motion state characteristic value of each animal; determining a characteristic value of agitation for each animal based on the number and frequency; The activity level of each animal in each sub-video segment is determined based on the motion state feature value and the anxiety feature value.

[0016] In another possible implementation, when the second determination module determines the environmental quality change curve of each animal cabin based on the environmental information, it is specifically used to: Dividing the temperature change curve into a plurality of temperature change segments according to preset intervals, dividing the humidity change curve into a plurality of humidity change segments, and dividing the oxygen content change curve into a plurality of oxygen content change segments; The ratio calculates the temperature average value of each temperature change section and calculates the absolute value of the first difference between each temperature average value and a preset temperature threshold value; Calculating the humidity average value of each humidity variation segment and calculating the absolute value of the second difference between each humidity average value and a preset humidity threshold value; Calculating an average oxygen content value in each oxygen content variation segment and calculating an absolute value of a third difference between each average oxygen content value and a preset humidity threshold value; The absolute value of the first difference, the absolute value of the second difference and the absolute value of the third difference with the same time interval are summed to obtain the environmental anomaly value of each time interval; An environmental quality change curve for each animal cabin is generated based on the environmental outliers at each time interval.

[0017] In another possible implementation, when the judgment module judges whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve, it is specifically used to: An approximate function is obtained by fitting based on the environmental quality change curve and the activity change curve; Calculating a first similarity between the approximate function and a preset function, wherein the preset function is determined according to environmental quality change curves and activity change curves of the remaining cabins except the target cabin; Determine a plurality of coordinate point groups with the same horizontal coordinate from the environmental quality change curve and the activity change curve, each coordinate point group including an environmental anomaly value and activity with the same horizontal coordinate; Mapping the plurality of coordinate point groups in a preset coordinate system to obtain a scatter plot of the plurality of coordinate point groups, wherein the abscissa of the preset coordinate system is the environmental anomaly value, and the ordinate is the activity degree; Determine the center point and the outline of the scatter plot; Calculating a distance between the center point and a center point of a preset contour map, and a second similarity between the contour and a contour of the preset contour map, wherein the preset contour map is determined based on a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curves and activity change curves of the remaining cabins except the target cabin; Determining the slope of the approximate function and the area within the contour range of the scatter plot; Determine a ratio of the slope to the area, and determine a difference between the ratio and a preset ratio, wherein the preset ratio is a ratio of a slope of a preset function to an area within a contour range of a preset contour graph; Determining an abnormality score for each target cabin based on the first similarity, the distance, the second similarity, and the difference; The target cabin whose abnormal score reaches a preset score threshold is determined as the abnormal cabin.

[0018] In another possible implementation, the system further comprises: A first fitting module is used to obtain an approximate function of each remaining cabin by fitting the environmental quality change curve and the activity change curve of each remaining cabin; The second fitting module is used to obtain the preset function by performing a quadratic fitting based on the approximate function of each remaining compartment.

[0019] In another possible implementation, the system determines the preset profile map, and further includes: A fourth determination module is used to determine a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curve and the activity change curve of each remaining cabin; A first mapping module, used for mapping a plurality of coordinate point groups of each remaining cabin into a preset coordinate system to obtain a scatter plot of each remaining cabin; A fifth determination module, configured to determine the center point of the scatter plot of each remaining compartment, and obtain the center point of the preset contour map by averaging the center points of the scatter plot of each remaining compartment; A second mapping module is used to determine the contour of the scatter plot of each remaining cabin, and to overlap and map the center point of the contour of the scatter plot of each remaining cabin to obtain the overlapping contour of the overlapping part of the contour of the scatter plot of all the remaining cabins; The completion module is used to complete the overlapping contours to obtain the preset contour map.

[0020] In a third aspect, the present application provides an electronic device, which adopts the following technical solution: An electronic device, comprising: at least one processor; Memory; At least one application, wherein at least one application is stored in a memory and configured to be executed by at least one processor, and at least one configuration is used to: execute a monitoring method for a multimodal laboratory environment as shown in any possible implementation of the first aspect.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium, when the computer program is executed in a computer, enables the computer to execute a multimodal laboratory environment monitoring method as described in any one of the first aspects.

[0022] In summary, the present application includes at least one of the following beneficial technical effects: Acquiring environmental information and monitoring videos facilitates subsequent analysis of whether each animal cabin is abnormal. The monitoring video records the specific activities of the animals in the animal cabin within a preset historical time period. Therefore, the activity change curve of each animal cabin can be accurately determined based on the monitoring video. The environmental information records the environmental changes in the animal cabin within a preset historical time period. Therefore, the environmental quality change curve of each animal cabin can be accurately determined based on the environmental information. The increase in animal activity in the cabin may be caused by a deterioration in the environment or by fighting between animals. Therefore, the target cabin whose activity reaches the preset activity threshold is first determined based on the activity change curve, and then the real abnormal cabin caused by the deterioration of environmental factors is comprehensively judged based on the environmental quality change curve and the activity change curve of the target cabin. After determining that there is an abnormal cabin, the ventilation device can be controlled to ventilate the abnormal cabin, thereby improving the environment in the abnormal cabin and making the environment in the abnormal cabin reach a level suitable for animal survival, thereby achieving the effect of facilitating timely discovery of abnormal cabins and improving the environment of the abnormal cabins. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flow chart of a monitoring method for a multimodal laboratory environment according to an embodiment of the present application.

[0024] Figure 2 It is a structural diagram of a monitoring system for a multimodal laboratory environment according to an embodiment of the present application.

[0025] Figure 3 It is a structural schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The present application is further described in detail below in conjunction with the accompanying drawings.

[0027] After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed, but such modifications are protected by patent law as long as they are within the scope of the claims of this application.

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0030] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0031] The embodiment of the present application provides a monitoring method for a laboratory environment based on multimodality, which is performed by an electronic device, which can be a server or a terminal device, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present application. Figure 1 As shown, the method includes step S101, step S102, step S103, step S104, step S105 and step S106, wherein: S101, obtaining environmental information of each animal cabin in the laboratory and surveillance video of each animal cabin in a preset historical time period.

[0032] The environmental information includes a temperature change curve, a humidity change curve, and an oxygen content change curve.

[0033] For the embodiment of the present application, the staff can install temperature sensors, humidity sensors and oxygen sensors in each animal cabin in advance, so as to collect the temperature value, humidity value and oxygen content in each animal cabin, and the server is connected to the temperature sensor, humidity sensor and oxygen sensor in communication, so that the server can obtain the real-time temperature value, real-time humidity value and real-time oxygen content in each animal cabin, and draw the temperature change curve, humidity change curve and oxygen content change curve according to the acquired data. The electronic device is connected to the server wirelessly, so that the environmental information within the preset historical time period can be obtained. The staff can also install a camera device in each animal cabin in advance, and the electronic device is connected to the camera device through a wire, so as to obtain the surveillance video of the animal cabin collected by each camera device. The preset historical time period can be the past half hour, the past hour, etc., which can be set by the staff through the visual operation interface and stored in the electronic device.

[0034] S102, determining an activity change curve of each animal cabin based on the monitoring video.

[0035] For the embodiment of the present application, the monitoring video records the specific circumstances of the animal activities in the animal cabin, so the electronic device can accurately determine the activity change curve of each animal cabin within a preset historical time period based on the monitoring video, and facilitate subsequent analysis of whether there is any abnormality in the animal cabin based on the activity change curve.

[0036] S103, determining an environmental quality change curve of each animal cabin based on the environmental information.

[0037] For the embodiment of the present application, after the electronic device obtains the environmental information, the environmental information records the multi-faceted and multi-modal environmental changes in the animal cabin, such as temperature and humidity. Therefore, the electronic device can determine the environmental quality change curve in each animal cabin based on the environmental information. The electronic device can facilitate subsequent analysis of whether there is any abnormality in the animal cabin based on the environmental quality change curve.

[0038] S104, determining a target cabin whose activity reaches a preset activity threshold based on the activity change curve.

[0039] With respect to the embodiments of the present application, when the temperature, humidity and other environmental conditions in the animal cabin make the animals uncomfortable, they will show behaviors such as restlessness and anxiety and want to escape from the animal cabin, that is, their activity level is high at this time. The high activity level of the animals in the animal cabin may also be caused by fighting among the animals. A preset activity threshold is used as a dividing point for whether the activity level is high. After the electronic device determines the activity change curve, it can calculate the average activity level on the activity change curve, and use the average activity level to characterize the overall level of the activity change curve. If the average activity level of an animal cabin reaches the preset activity threshold, it means that the animals in the animal cabin have been too active during a preset historical time period, which may be caused by environmental factors in the cabin, but may also be caused by fighting among the animals in the cabin. Therefore, the electronic device first determines the target cabin where the animal activity level is too high.

[0040] S105, judging whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve.

[0041] For the embodiment of the present application, after the electronic device determines the target cabin, it is necessary to further screen the target cabin to determine the abnormal cabin caused by environmental factors in the cabin. Therefore, the electronic device conducts a comprehensive analysis and judgment based on the environmental quality change curve and the activity change curve to determine the real abnormal cabin caused by environmental factors.

[0042] S106: If there is an abnormal cabin, control the ventilation device to ventilate the abnormal cabin.

[0043] For the embodiment of the present application, the staff can install a ventilation device in each animal cabin, and the ventilation device has functions such as air heating, air cooling, and air humidification. After the electronic device determines that there is an abnormal cabin, the electronic device can determine an accurate adjustment strategy based on the environmental information of the abnormal cabin. Specifically, the electronic device can input the environmental information of the abnormal cabin into the trained network model for strategy calculation, thereby determining the ventilation adjustment strategy in the abnormal cabin, and then the electronic device controls the ventilation device in the abnormal cabin to ventilate the abnormal cabin according to the determined ventilation adjustment strategy, so that the environment in the abnormal cabin reaches the standard environment for animal survival, and achieves the effect of facilitating the timely discovery of abnormal cabins and improving the environment of the abnormal cabins. If there is no abnormal cabin, the electronic device does not perform any operation and performs a new round of monitoring.

[0044] In a possible implementation of the embodiment of the present application, in step S102, determining the activity change curve of each animal cabin based on the monitoring video includes step S1021 (not shown in the figure), step S1022 (not shown in the figure), step S1023 (not shown in the figure), step S1024 (not shown in the figure), step S1025 (not shown in the figure), step S1026 (not shown in the figure) and step S1027 (not shown in the figure), wherein: S1021, dividing the surveillance video of any animal cabin into a plurality of sub-video segments according to preset intervals.

[0045] In the embodiment of the present application, for any animal cabin, the electronic device divides the surveillance video of the animal cabin into multiple sub-video segments, and the sub-video segments are divided according to preset intervals, which can be a three-minute time interval, a five-minute time interval, etc. Dividing the surveillance video into multiple sub-video segments facilitates the subsequent determination of a more accurate activity change curve.

[0046] S1022, determining the leg features, nose tip features, and body center features of each animal in any animal cabin from each sub-video segment.

[0047] For the embodiment of the present application, the electronic device inputs each sub-video into the trained network model for feature recognition, thereby identifying the leg features, nose tip features and body center features of each animal. Animals move with their legs, so the leg features can characterize the activity of the animal to a certain extent. The leg features can be any leg of each animal. Animals also explore the surrounding environment with their sense of smell and vision (such as mice, rabbits, etc.). When exploring with head organs such as sense of smell, the animal's head will also move, so the nose tip features can also characterize the activity of the animal to a certain extent. The body center characterizes the position of the animal in the cabin, and the change in the position of the animal in the cabin can also characterize the activity of the animal to a certain extent.

[0048] Specifically, the network model can be a convolutional neural network model, a recurrent neural network model, or other types of network models, which are not limited here.

[0049] S1023, determining a first motion trajectory of the leg features, a second motion trajectory of the nose tip features, and a third motion trajectory of the body center point features.

[0050] For the embodiment of the present application, after the electronic device determines the leg features, it visually tracks the leg features in the surveillance video to obtain the first motion trajectory of the leg features of each animal. Similarly, the nose tip features in the surveillance video are visually tracked to obtain the second motion trajectory of the nose tip features of each animal, and the body center point features in the surveillance video are visually tracked to obtain the third motion trajectory of each animal. The longer the first motion trajectory, the second motion trajectory, and the third motion trajectory, the more frequent the animal's activities and the higher its activity level.

[0051] S1024, determining the number and frequency of each animal's nose tip feature being located on the side wall of any animal's cabin based on each sub-video segment.

[0052] For the embodiment of the present application, the electronic device also performs feature recognition on the side walls of the animal cabin, thereby obtaining the edge side wall features of the animal cabin. Specifically, edge detection can be performed on the monitoring video. First, the monitoring video is denoised to obtain a denoised monitoring video, and then the denoised monitoring video is grayscale transformed to obtain a grayscale monitoring video. The location where the grayscale value in the monitoring video undergoes a step change is the side wall of the animal cabin. The electronic device determines the side wall range of each animal cabin in the monitoring video, and then determines whether the nose tip feature of each animal is within the range of the side wall. If so, the time point when it is within the side wall range is counted and stored. The electronic device can determine the frequency based on the time point each time the nose tip feature is within the side wall range. The higher the frequency and the more times, the more active and anxious the animal is, and it wants to escape from the animal cabin.

[0053] S1025 , determining the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times and the frequency.

[0054] For the embodiments of the present application, in summary, the first motion trajectory, the second motion trajectory, the third motion trajectory, and the number and frequency of the nose tip feature located on the side wall of the animal cabin are all key factors affecting the activity of each animal in the sub-video segment. Therefore, the electronic device can comprehensively determine the accurate activity of each animal in each sub-video based on the above five factors.

[0055] S1026, averaging the activity of each animal in each sub-video segment to obtain the sub-activity corresponding to each sub-video segment.

[0056] For the embodiment of the present application, after the electronic device determines the activity level of each animal in each sub-video segment, it calculates the sub-activity level corresponding to each sub-video segment through the average value calculation formula, that is, the overall activity level of the entire animal cabin in the time period corresponding to the sub-video segment.

[0057] S1027, generating an activity change curve of any cabin based on the sub-activity.

[0058] For the embodiment of the present application, after the electronic device determines the sub-activity of each sub-video segment, it determines the time point corresponding to the sub-video segment. For example, the time point corresponding to the middle point of the sub-video segment can be determined as the horizontal coordinate, and the corresponding sub-activity can be determined as the vertical coordinate, thereby obtaining multiple coordinate points. Then, the electronic device connects the multiple coordinate points in sequence to obtain an activity change curve. It should be noted that the electronic device determines the activity change curve for each animal cabin in accordance with steps S1021 to S1027.

[0059] In a possible implementation of the embodiment of the present application, in step S1025, the activity of each animal in each sub-video segment is determined based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times and the frequency, specifically including step Sa (not shown in the figure), step Sb (not shown in the figure), step Sc (not shown in the figure) and step Sd (not shown in the figure), wherein: Sa, determine the total length of the first motion trajectory and the second motion trajectory.

[0060] For the embodiment of the present application, the electronic device can determine the number of pixels corresponding to the first motion trajectory and the second motion trajectory, and then use a preset scale to perform proportional amplification calculation to obtain the length of the first motion trajectory and the length of the second motion trajectory. The preset scale is determined by the staff based on the size of the animal cabin in the monitoring video, so that true and accurate data can be obtained in the subsequent calculation of the length of the first motion trajectory. After the electronic device determines the length of the first motion trajectory and the length of the second motion trajectory, the total length can be obtained by summing them up. The motion trajectory of the legs and the motion trajectory of the nose tip are the main activities of the animal crawling and sniffing. The longer the motion trajectory of the main activity behavior, the more active it is.

[0061] Sb, determines the product of the total length and the length of the third motion trajectory.

[0062] Among them, the product represents the characteristic value of the movement state of each animal.

[0063] For the embodiment of the present application, the third motion track represents the position movement of each animal. The more frequent the position movement, the more active the animal. Therefore, the electronic device calculates the product of the total length of the first motion track and the second motion track and the length of the third motion track. The larger the product, the higher the animal activity and the higher the motion state. Therefore, the product represents the motion state of each animal, that is, the motion state characteristic value.

[0064] Sc, determine the characteristic value of agitation for each animal based on the number and frequency.

[0065] For the embodiment of the present application, the more times the nose tip feature is located on the side wall of the cabin and the higher the frequency of the nose tip feature touching the side wall of the cabin, the more the animal wants to escape from the cabin and the more anxious the animal is. Therefore, the staff can set the coefficients corresponding to the number of times and the frequency in advance and store them in the electronic device. After the electronic device determines the number of times and the frequency, it calls the corresponding coefficients for weighted calculation to determine the anxiety characteristic value of each animal in the sub-video segment.

[0066] Sd, determines the activity of each animal in the sub-video segment based on the motion state feature value and the anxiety feature value.

[0067] For the embodiments of the present application, in summary, the motion state characteristic value and the anxiety characteristic value are both key factors in characterizing the activity of each animal, so the electronic device can sum the motion state characteristic value and the anxiety characteristic value to obtain the activity. Or the staff sets the coefficients corresponding to the motion state characteristic value and the anxiety characteristic value in advance, and then the electronic device performs weighted calculation according to the corresponding coefficients to obtain the activity. It is more accurate to comprehensively determine the activity by the motion trajectory of the key characteristics of each animal and the anxiety characteristic value that characterizes the anxious intention to escape from the cabin.

[0068] In a possible implementation of the embodiment of the present application, in step S103, the environmental quality change curve of each animal cabin is determined based on the environmental information, specifically including step S1031 (not shown in the figure), step S1032 (not shown in the figure), step S1033 (not shown in the figure), step S1034 (not shown in the figure), step S1035 (not shown in the figure) and step S1036 (not shown in the figure), wherein: S1031, dividing the temperature change curve into a plurality of temperature change segments according to preset intervals, dividing the humidity change curve into a plurality of humidity change segments, and dividing the oxygen content change curve into a plurality of oxygen content change segments.

[0069] For the embodiment of the present application, the preset interval is consistent with the preset interval in step S1021. The electronic device divides the temperature change curve, the humidity change curve and the oxygen content change curve according to the same preset interval to obtain corresponding data segments, i.e., multiple temperature change segments, multiple humidity change segments and multiple oxygen content change segments, so as to facilitate the subsequent analysis of abnormalities in combination with the activity change curve to maintain the consistency of the data in time span.

[0070] S1032, calculating the average temperature value of each temperature change section by ratio and calculating the absolute value of the first difference between each average temperature value and a preset temperature threshold.

[0071] For the embodiment of the present application, the electronic device calculates the temperature average value through the average value calculation formula according to the temperature value of each temperature change segment, uses the temperature average value to represent the overall temperature of each temperature change segment, and then calculates the absolute value of the first difference between each temperature average value and the preset temperature threshold value, the preset temperature threshold value represents the standard ambient temperature for animal survival, and the absolute value of the first difference represents the degree of deviation between the temperature of each temperature change segment and the preset temperature threshold value. The larger the absolute value of the first difference, the greater the temperature deviation.

[0072] S1033, calculating the humidity average value of each humidity change segment and calculating the absolute value of the second difference between each humidity average value and a preset humidity threshold.

[0073] For the embodiment of the present application, the electronic device calculates the humidity average value according to the humidity value of each humidity change segment through the average value calculation formula, uses the humidity average value to represent the overall humidity of each humidity change segment, and then calculates the absolute value of the second difference between each humidity average value and the preset humidity threshold value, the preset humidity threshold value represents the standard environmental humidity for animal survival, and the absolute value of the second difference represents the degree of deviation between the humidity of each humidity change segment and the preset humidity threshold value. The larger the absolute value of the second difference, the greater the humidity deviation.

[0074] S1034, calculating the average oxygen content of each oxygen content variation segment and calculating the absolute value of the third difference between each average oxygen content and a preset humidity threshold.

[0075] For the embodiment of the present application, the electronic device calculates the average oxygen content according to the oxygen content of each oxygen content change segment through the average value calculation formula, uses the average oxygen content to characterize the overall oxygen content of each oxygen content change segment, and then calculates the absolute value of the third difference between each oxygen content average value and the preset oxygen content threshold value, the preset oxygen content threshold value characterizes the standard environment oxygen content for animal survival, and the absolute value of the third difference characterizes the degree of deviation between the oxygen content of each oxygen content change segment and the preset oxygen content threshold value. The larger the absolute value of the third difference, the greater the oxygen content deviation.

[0076] S1035: Sum the absolute value of the first difference, the absolute value of the second difference, and the absolute value of the third difference with the same time interval to obtain an environmental anomaly value for each time interval.

[0077] For the embodiment of the present application, the absolute value of the first difference, the absolute value of the second difference and the absolute value of the third difference are all key factors representing the quality of the environment. Therefore, the electronic device can sum the absolute value of the first difference, the absolute value of the second difference and the absolute value of the third difference to obtain the environmental anomaly value representing each time interval. The larger the environmental anomaly value, the more abnormal the environment.

[0078] S1036, generating an environmental quality change curve for each animal cabin based on the environmental abnormality value at each time interval.

[0079] For the embodiments of the present application, after the electronic device determines the sub-environment anomaly value of each time interval, it determines the time point corresponding to each time interval. For example, the time point corresponding to the middle point of the time interval can be determined as the horizontal coordinate, and the corresponding environmental anomaly value can be determined as the vertical coordinate, thereby obtaining multiple coordinate points. Then, the electronic device can connect the multiple coordinate points in sequence to obtain the living environment quality change curve.

[0080] In a possible implementation of the embodiment of the present application, in step S105, judging whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve specifically includes step 1, step 2, step 3, step 4, step 5, step 6, step 7, step 8, step 9 and step 10, wherein: Step 1: Get an approximate function by fitting based on the environmental quality change curve and the activity change curve.

[0081] For the embodiment of the present application, the electronic device can perform linear fitting based on the coordinate points on the environmental quality change curve to obtain a linear function corresponding to the environmental quality change curve, perform linear fitting based on the activity change curve to obtain a linear function about the activity change curve, and then average or weighted average the coefficients of the two linear functions to obtain an approximate function. Alternatively, the electronic device directly maps the coordinate points on the environmental quality change curve and the activity change curve in the same coordinate system, and then performs linear fitting on the coordinate points in the coordinate system to obtain an approximate function.

[0082] Step 2: Calculate the first similarity between the approximate function and the preset function.

[0083] Among them, the preset function is determined based on the environmental quality change curve and activity change curve of the remaining cabins except the target cabin.

[0084] For the embodiment of the present application, the remaining cabins are all cabins where the activity of animals is normal, which means that the environment in the remaining cabins is normal, that is, the temperature, humidity and oxygen content meet the survival requirements of animals. The electronic device calculates the first similarity between the approximate function and the preset function, specifically by calculating the Euclidean distance or cosine similarity between the approximate function and the preset function. Taking cosine similarity as an example, the closer the cosine similarity is to 1, the more similar it is, which means that the environment of the target cabin is more normal.

[0085] Step three, determine multiple coordinate point groups with the same horizontal coordinates from the environmental quality change curve and the activity change curve, each coordinate point group includes environmental anomaly values ​​and activity with the same horizontal coordinates.

[0086] For the embodiment of the present application, the horizontal coordinates of the environment quality change curve and the activity change curve are the same, so the electronic device determines a coordinate point group for each environmental anomaly value and activity with the same horizontal coordinate.

[0087] Step 4: Map the multiple coordinate point groups in a preset coordinate system to obtain a scatter plot of the multiple coordinate point groups.

[0088] Among them, the horizontal axis of the preset coordinate system is the environmental anomaly value, and the vertical axis is the activity.

[0089] For the embodiment of the present application, after the electronic device determines the multiple coordinate point groups corresponding to each target cabin, the multiple coordinate point groups are mapped in a preset coordinate system with the horizontal axis being the environmental anomaly value and the vertical axis being the activity level, thereby obtaining a scatter plot. The scatter plot represents the distribution of the relationship between the environmental anomaly value and the activity level in the same time period or the same time period.

[0090] Step 5: Determine the center point and outline of the scatter plot.

[0091] For the embodiment of the present application, after the electronic device determines the scatter plot of each target cabin, the horizontal coordinates and vertical coordinates of all coordinate points of the scatter plot are averaged to obtain the center point coordinates, and then the electronic device sequentially connects the outermost coordinates of the scatter plot to obtain the outline of the scatter plot. The center point and outline of the scatter plot are used as the key features of each scatter plot.

[0092] Step six, calculating the distance between the center point and the center point of the preset contour map, and the second similarity between the contour and the contour of the preset contour map.

[0093] The preset contour map is determined based on a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curves and activity change curves of the remaining cabins except the target cabin.

[0094] For the embodiment of the present application, the preset contour map is also determined by using multiple coordinate point groups of the remaining cabins. Since the remaining cabins are cabins with normal animal activity, it means that the environment in the remaining cabins is normal, that is, the temperature, humidity and oxygen content meet the survival requirements of animals. Therefore, the preset contour map determined based on the remaining cabins represents the relationship between the environmental abnormality value and the activity level that the cabin environment is normal and meets the survival requirements of organisms. That is, the relationship between the environmental abnormality value and the activity level meets the preset contour map. Figure 1 If the values ​​of the target cabin environment are consistent or very similar, it means that the abnormal value and activity level of the target cabin environment may meet the standard for animal survival. The electronic device can calculate the distance between the center point and the center point of the preset contour map through the distance formula between two points. This distance represents the gap between the scatter plot and the preset contour map to a certain extent.

[0095] Step seven, determine the slope of the approximate function and the area within the contour range of the scatter plot.

[0096] For the embodiment of the present application, after the electronic device determines the approximate function, it can determine the slope of the approximate function. After the electronic device determines the scatter plot, it can determine the number of complete squares within the outline of the scatter plot, then splice the number of incomplete squares to obtain the number of spliced ​​squares, and finally sum them up to obtain the area within the outline of the scatter plot.

[0097] Step eight, determining the ratio of the slope to the area, and determining the difference between the ratio and a preset ratio.

[0098] The preset ratio is the ratio of the slope of the preset function to the area within the contour range of the preset contour graph.

[0099] For the embodiment of the present application, the electronic device calculates the ratio of the slope to the area, and uses the ratio to characterize the relationship between the approximate function and the scatter plot. Similarly, the electronic device determines in advance the ratio of the slope of the preset function to the area of ​​the preset contour graph, that is, the preset ratio, and then subtracts the ratio of the slope to the area from the preset ratio to obtain a difference, which characterizes the gap between the ratio corresponding to the target cabin and the preset ratio.

[0100] Step nine: determining an abnormality score of each target cabin based on the first similarity, the distance, the second similarity, and the difference.

[0101] For the embodiment of the present application, the first similarity, distance, second similarity and difference are all key factors that characterize the abnormality of each target cabin, and the proportion of the degree of influence is different, so the staff can set the coefficients corresponding to the first similarity, distance, second similarity and difference in advance, and then the electronic device calls the corresponding coefficients for weighted calculation to obtain the abnormality score of each target cabin. In order to facilitate calculation and ensure the logic of the abnormality score, the electronic device can use the reciprocal of the first similarity, the reciprocal of the second similarity combined with the distance and the difference for weighted calculation.

[0102] Step ten, determining the target cabin whose abnormal score reaches a preset score threshold as an abnormal cabin.

[0103] For the embodiment of the present application, the preset score threshold is used as the dividing point of whether the abnormal score is too high. After the electronic device determines the abnormal score of each target cabin, it compares it with the preset score threshold respectively, so as to determine the target cabins whose abnormal scores reach the preset score threshold. These cabins are abnormal cabins. It is more accurate to comprehensively determine the abnormal cabins by determining the first similarity, the scatter plot, the second similarity, etc.

[0104] A possible implementation of the embodiment of the present application is to determine the preset function, specifically including step S107 (not shown in the figure) and step S108 (not shown in the figure), wherein: S107, performing fitting based on the environmental quality change curve and activity change curve of each remaining cabin to obtain an approximate function of each remaining cabin.

[0105] S108, performing quadratic fitting based on the approximate function of each remaining compartment to obtain a preset function.

[0106] For the embodiments of the present application, the electronic device can perform linear fitting based on the coordinate points on the environmental quality change curve of each remaining cabin to obtain a linear function corresponding to the environmental quality change curve, perform linear fitting based on the activity change curve to obtain a linear function about the activity change curve, and then average or weighted average the coefficients of the two linear functions to obtain an approximate function for each remaining cabin. Alternatively, the electronic device directly maps the coordinate points on the environmental quality change curve and the activity change curve of each remaining cabin in the same coordinate system, and then performs linear fitting on the coordinate points in the coordinate system to obtain an approximate function for each remaining cabin. The electronic device then maps all the approximate functions of each remaining cabin to the same coordinate system and performs quadratic fitting to obtain a preset function generated for all the remaining cabins. The preset function obtained by fitting in this way is more accurate.

[0107] A possible implementation of the embodiment of the present application is to determine a preset contour map, specifically including step S1 (not shown in the figure), step S2 (not shown in the figure), step S3 (not shown in the figure), step S4 (not shown in the figure) and step S5 (not shown in the figure), wherein: S1, determining a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curve and the activity change curve of each remaining cabin.

[0108] For the embodiment of the present application, the horizontal coordinates of the environmental quality change curve and the activity change curve of the remaining cabins are the same, so the electronic device determines multiple coordinate point groups for the environmental anomaly values ​​and activity levels with the same horizontal coordinates for each remaining cabin.

[0109] S2, mapping the multiple coordinate point groups of each remaining cabin into a preset coordinate system to obtain a scatter plot of each remaining cabin.

[0110] For the embodiment of the present application, after the electronic device determines the multiple coordinate point groups of each remaining cabin, it maps the multiple coordinate point groups of each remaining cabin to a preset coordinate system with the horizontal axis being the environmental outlier value and the vertical axis being the activity level, thereby obtaining a scatter plot for each remaining cabin.

[0111] S3, determining the center point of the scatter plot of each remaining cabin, and obtaining the center point of the preset contour map by averaging the center point of the scatter plot of each remaining cabin.

[0112] For the embodiment of the present application, the electronic device obtains the center point of each scatter plot of each remaining compartment by calculating the average value of all horizontal coordinate points and the average value of all vertical coordinate points, and then obtains the center point coordinates of the scatter plots of all remaining compartments by averaging the horizontal and vertical coordinates to obtain the center point coordinates of the preset contour diagram.

[0113] S4, determining the contour of the scatter diagram of each remaining cabin, and performing coincidence mapping on the center point of the contour of the scatter diagram of each remaining cabin, to obtain the overlapping contour of the overlapping part of the contours of the scatter diagrams of all the remaining cabins.

[0114] For the embodiment of the present application, the electronic device determines the outline of the scatter plot of each remaining cabin in the manner of step five, and then overlaps the center points of all the remaining cabin scatter plots, thereby completing the overlap mapping of the outlines of all the remaining cabin scatter plots, and then the electronic device can determine the overlapping part of the outlines of all the remaining cabin scatter plots, that is, the overlapping outline.

[0115] S5, completing the overlapping contours by connecting lines to obtain a preset contour map.

[0116] For the embodiment of the present application, after the electronic device determines the overlapping contours, it completes the disconnected areas according to the endpoints of the adjacent overlapping contours to obtain a preset contour map. The preset contour map is determined comprehensively according to the contours of the scatter plots of all the remaining cabins, thereby making the preset contour map more accurate and better able to characterize the relationship between environmental anomalies and activity levels that meet the animal survival requirements.

[0117] The above embodiment introduces a monitoring method based on a multimodal laboratory environment from the perspective of method flow. The following embodiment introduces a monitoring system based on a multimodal laboratory environment from the perspective of a virtual module or a virtual unit. For details, please refer to the following embodiment.

[0118] The present application embodiment provides a monitoring system 20 based on a multi-modal laboratory environment, such as Figure 2 As shown, the monitoring system 20 based on the multimodal laboratory environment may specifically include: The data acquisition module 201 is used to obtain the environmental information of each animal cabin in the laboratory and the monitoring video of each animal cabin in a preset historical time period, and the environmental information includes a temperature change curve, a humidity change curve and an oxygen content change curve; A first determination module 202, for determining an activity change curve of each animal cabin based on the monitoring video; A second determination module 203 is used to determine the environmental quality change curve of each animal cabin based on the environmental information; A third determination module 204 is used to determine a target cabin whose activity reaches a preset activity threshold based on the activity change curve; A judgment module 205, for judging whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve; The control module 206 is used to control the ventilation device to ventilate the abnormal cabin when there is an abnormal cabin.

[0119] The embodiment of the present application discloses a monitoring system 20 for a laboratory environment based on a multimodal system, wherein a data acquisition module 201 acquires environmental information and monitoring videos to facilitate subsequent analysis of whether each animal cabin is abnormal. The monitoring videos record the specific activities of the animals in the animal cabins within a preset historical time period. Therefore, a first determination module 202 can accurately determine the activity change curve of each animal cabin based on the monitoring videos. The environmental information records the environmental changes of the animal cabins within a preset historical time period. Therefore, a second determination module 203 can accurately determine the environmental quality change curve of each animal cabin based on the environmental information. The increase in the activity of the animals in the cabin may be due to It may be caused by environmental deterioration or fighting between animals. Therefore, the third determination module 204 first determines the target cabin whose activity reaches the preset activity threshold based on the activity change curve. The judgment module 205 then comprehensively determines the real abnormal cabin caused by the deterioration of environmental factors based on the environmental quality change curve and the activity change curve of the target cabin. After determining that there is an abnormal cabin, the control module 206 controls the ventilation device to ventilate the abnormal cabin, thereby improving the environment in the abnormal cabin and making the environment in the abnormal cabin reach a level suitable for animal survival, thereby achieving the effect of facilitating timely detection of abnormal cabins and improving the environment of abnormal cabins.

[0120] In a possible implementation of the embodiment of the present application, when the first determination module 202 determines the activity change curve of each animal cabin based on the monitoring video, it is specifically used to: Splitting the surveillance video of any animal cabin into a plurality of sub-video segments according to preset intervals; Determine the leg features, nose tip features and body center features of each animal in any animal cabin from each sub-video segment; Determine a first motion trajectory of the leg features, a second motion trajectory of the nose tip features, and a third motion trajectory of the body center point features; determining the number of times and the frequency at which the nose tip feature of each animal is located on the side wall of any of the animal's compartments based on each of the sub-video segments; Determine the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times and the frequency; The activity level of each animal in each sub-video segment is averaged to obtain the sub-activity level corresponding to each sub-video segment; Generate an activity change curve of any cabin based on the sub-activity.

[0121] In a possible implementation of the embodiment of the present application, when the first determination module 202 determines the activity of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times and the frequency, it is specifically used to: Determining the total length of the first motion trajectory and the second motion trajectory; determining the product of the total length and the length of the third motion trajectory, the product representing a characteristic value of the motion state of each animal; Determine the agitation characteristic value for each animal based on the number and frequency; The activity level of each animal in each sub-video segment is determined based on the motion state feature value and the restlessness feature value.

[0122] In a possible implementation of the embodiment of the present application, when the second determination module 203 determines the environmental quality change curve of each animal cabin based on the environmental information, it is specifically used to: The temperature change curve is divided into a plurality of temperature change segments according to preset intervals, the humidity change curve is divided into a plurality of humidity change segments, and the oxygen content change curve is divided into a plurality of oxygen content change segments; The ratio calculates the temperature average value of each temperature change section and calculates the absolute value of the first difference between each temperature average value and a preset temperature threshold value; Calculating the humidity average value of each humidity variation segment and calculating the absolute value of the second difference between each humidity average value and a preset humidity threshold value; Calculating an average oxygen content value in each oxygen content variation segment and calculating an absolute value of a third difference between each average oxygen content value and a preset humidity threshold value; The absolute value of the first difference, the absolute value of the second difference and the absolute value of the third difference with the same time interval are summed to obtain the environmental anomaly value of each time interval; An environmental quality change curve for each animal cabin is generated based on the environmental outliers at each time interval.

[0123] In a possible implementation of the embodiment of the present application, when the judgment module 205 judges whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve, it is specifically used to: An approximate function is obtained by fitting based on the environmental quality change curve and the activity change curve; Calculating a first similarity between the approximate function and a preset function, where the preset function is determined based on environmental quality change curves and activity change curves of the remaining cabins except the target cabin; Determine multiple coordinate point groups with the same horizontal coordinate from the environmental quality change curve and the activity change curve, each coordinate point group includes environmental abnormal values ​​and activity with the same horizontal coordinate; Mapping the multiple coordinate point groups in a preset coordinate system to obtain scatter plots of the multiple coordinate point groups, where the abscissa of the preset coordinate system is the environmental outlier value and the ordinate is the activity level; Determine the center point and outline of the scatter plot; Calculating the distance between the center point and the center point of the preset contour map, and the second similarity between the contour and the contour of the preset contour map, wherein the preset contour map is determined based on a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curves and activity change curves of the remaining cabins except the target cabin; Determine the slope of the approximate function and the area within the contour of the scatter plot; Determine a ratio of the slope to the area, and determine a difference between the ratio and a preset ratio, wherein the preset ratio is a ratio of the slope of a preset function to an area within a contour range of a preset contour graph; Determining an abnormality score for each target cabin based on the first similarity, the distance, the second similarity, and the difference; The target cabin whose abnormal score reaches the preset score threshold is determined as an abnormal cabin.

[0124] In a possible implementation manner of the embodiment of the present application, the preset function is determined, and the system 20 further includes: A first fitting module is used to obtain an approximate function of each remaining cabin by fitting the environmental quality change curve and the activity change curve of each remaining cabin; The second fitting module is used to obtain a preset function by performing a quadratic fitting based on the approximate function of each remaining compartment.

[0125] In a possible implementation of the embodiment of the present application, the system 20 determines a preset profile map and further includes: A fourth determination module is used to determine a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curve and the activity change curve of each remaining cabin; A first mapping module, used for mapping a plurality of coordinate point groups of each remaining cabin into a preset coordinate system to obtain a scatter plot of each remaining cabin; A fifth determination module, used to determine the center point of the scatter plot of each remaining cabin, and obtain the center point of the preset contour map by averaging the center points of the scatter plot of each remaining cabin; A second mapping module is used to determine the contour of the scatter plot of each remaining cabin, and to overlap and map the center point of the contour of the scatter plot of each remaining cabin to obtain the overlapping contour of the overlapping part of the contour of the scatter plot of all the remaining cabins; The completion module is used to complete the overlapping contours to obtain a preset contour map.

[0126] Technicians in the relevant field can clearly understand that, for the convenience and simplicity of description, the specific working process of the monitoring system 20 based on a multimodal laboratory environment described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0127] An electronic device is provided in an embodiment of the present application, such as Figure 3 As shown, Figure 3 The electronic device 30 shown includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 30 may also include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one, and the structure of the electronic device 30 does not constitute a limitation on the embodiments of the present application.

[0128] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0129] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but it does not mean that there is only one bus or only one type of bus.

[0130] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0131] The memory 303 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the contents shown in the above method embodiment.

[0132] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 3 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0133] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content in the aforementioned method embodiment. Compared with the related art, the environmental information and monitoring video obtained in the embodiment of the present application facilitate the subsequent analysis of whether each animal cabin is abnormal. The monitoring video records the specific activities of the animals in the animal cabin within a preset historical time period. Therefore, the activity change curve of each animal cabin can be accurately determined based on the monitoring video. The environmental information records the environmental changes in the animal cabin within a preset historical time period. Therefore, the environmental quality change curve of each animal cabin can be accurately determined based on the environmental information. The increase in the activity of the animals in the cabin may be caused by the deterioration of the environment or by the animals fighting with each other. Therefore, the target cabin whose activity reaches the preset activity threshold is first determined based on the activity change curve, and then the real abnormal cabin caused by the deterioration of environmental factors is comprehensively judged based on the environmental quality change curve and the activity change curve of the target cabin. After determining that there is an abnormal cabin, the ventilation device can be controlled to ventilate the abnormal cabin, thereby improving the environment in the abnormal cabin and making the environment in the abnormal cabin reach a level suitable for animal survival, thereby achieving the effect of facilitating timely discovery of abnormal cabins and improving the environment of the abnormal cabins.

[0134] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0135] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A multimodal laboratory environment monitoring method, characterized in that: include: Obtaining environmental information of each animal cabin in the laboratory during a preset historical time period and surveillance video of each animal cabin, wherein the environmental information includes a temperature change curve, a humidity change curve, and an oxygen content change curve; Determine an activity change curve of each animal cabin based on the monitoring video; Determine an environmental quality change curve for each animal cabin based on the environmental information; Determining a target cabin whose activity reaches a preset activity threshold based on the activity change curve; Determining whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve; If there is an abnormal compartment, the ventilation device is controlled to ventilate the abnormal compartment.

2. The method for monitoring a laboratory environment based on multimodality according to claim 1, characterized in that: The determining of the activity change curve of each animal cabin based on the monitoring video includes: Splitting the surveillance video of any animal cabin into a plurality of sub-video segments according to preset intervals; Determine the leg features, nose tip features and body center features of each animal in any one of the animal cabins from each sub-video segment; Determine a first motion trajectory of the leg feature, a second motion trajectory of the nose tip feature, and a third motion trajectory of the body center point feature; Determining the number of times and the frequency at which the nose tip feature of each animal is located on the side wall of any of the animal compartments based on each of the sub-video segments; Determine the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times and the frequency; The activity level of each animal in each sub-video segment is averaged to obtain the sub-activity level corresponding to each sub-video segment; An activity change curve of any cabin is generated based on the sub-activity.

3. The method for monitoring a laboratory environment based on multimodality according to claim 2, characterized in that: The determining the activity level of each animal in each sub-video segment based on the first motion trajectory, the second motion trajectory, the third motion trajectory, the number of times and the frequency includes: Determining the total length of the first motion trajectory and the second motion trajectory; Determine the product of the total length and the length of the third motion trajectory, wherein the product represents a motion state characteristic value of each animal; determining a characteristic value of agitation for each animal based on the number and frequency; The activity level of each animal in each sub-video segment is determined based on the motion state feature value and the anxiety feature value.

4. The method for monitoring a laboratory environment based on multimodality according to claim 1, characterized in that: The step of determining the environmental quality change curve of each animal cabin based on the environmental information comprises: Dividing the temperature change curve into a plurality of temperature change segments according to preset intervals, dividing the humidity change curve into a plurality of humidity change segments, and dividing the oxygen content change curve into a plurality of oxygen content change segments; The ratio calculates the temperature average value of each temperature change section and calculates the absolute value of the first difference between each temperature average value and a preset temperature threshold value; Calculating the humidity average value of each humidity variation segment and calculating the absolute value of the second difference between each humidity average value and a preset humidity threshold value; Calculating an average oxygen content value in each oxygen content variation segment and calculating an absolute value of a third difference between each average oxygen content value and a preset humidity threshold value; The absolute value of the first difference, the absolute value of the second difference and the absolute value of the third difference with the same time interval are summed to obtain the environmental anomaly value of each time interval; An environmental quality change curve for each animal cabin is generated based on the environmental outliers at each time interval.

5. The method for monitoring a laboratory environment based on multimodality according to claim 1, characterized in that: The determining whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve includes: An approximate function is obtained by fitting based on the environmental quality change curve and the activity change curve; Calculating a first similarity between the approximate function and a preset function, wherein the preset function is determined according to environmental quality change curves and activity change curves of the remaining cabins except the target cabin; Determine a plurality of coordinate point groups with the same horizontal coordinate from the environmental quality change curve and the activity change curve, each coordinate point group including an environmental anomaly value and activity with the same horizontal coordinate; Mapping the plurality of coordinate point groups in a preset coordinate system to obtain a scatter plot of the plurality of coordinate point groups, wherein the abscissa of the preset coordinate system is the environmental anomaly value, and the ordinate is the activity degree; Determine the center point and the outline of the scatter plot; Calculating a distance between the center point and a center point of a preset contour map, and a second similarity between the contour and a contour of the preset contour map, wherein the preset contour map is determined based on a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curves and activity change curves of the remaining cabins except the target cabin; Determining the slope of the approximate function and the area within the contour range of the scatter plot; Determine a ratio of the slope to the area, and determine a difference between the ratio and a preset ratio, wherein the preset ratio is a ratio of a slope of a preset function to an area within a contour range of a preset contour graph; Determining an abnormality score for each target cabin based on the first similarity, the distance, the second similarity, and the difference; The target cabin whose abnormal score reaches a preset score threshold is determined as the abnormal cabin.

6. The method for monitoring a laboratory environment based on multimodality according to claim 5, characterized in that: Determining the preset function includes: An approximate function of each remaining cabin is obtained by fitting the environmental quality change curve and activity change curve of each remaining cabin; The preset function is obtained by performing quadratic fitting based on the approximate function of each remaining compartment.

7. The method for monitoring a laboratory environment based on multimodality according to claim 5, characterized in that: Determining the preset contour map includes: Determine a plurality of coordinate point groups with the same horizontal coordinates in the environmental quality change curve and the activity change curve of each remaining cabin; Mapping multiple coordinate point groups of each remaining cabin into a preset coordinate system to obtain a scatter plot of each remaining cabin; Determine the center point of the scatter plot of each remaining compartment, and obtain the center point of the preset contour map by averaging the center points of the scatter plot of each remaining compartment; Determine the contour of the scatter plot of each remaining cabin, and perform coincidence mapping on the center point of the contour of the scatter plot of each remaining cabin to obtain the overlapping contour of the overlapping part of the contours of the scatter plots of all the remaining cabins; The overlapping contours are connected and completed to obtain the preset contour map.

8. A monitoring system for a laboratory environment based on multimodality, characterized in that: include: A data acquisition module, used to acquire environmental information of each animal cabin in the laboratory during a preset historical period and a monitoring video of each animal cabin, wherein the environmental information includes a temperature change curve, a humidity change curve, and an oxygen content change curve; A first determination module, configured to determine an activity change curve of each animal cabin based on the monitoring video; A second determination module is used to determine an environmental quality change curve of each animal cabin based on the environmental information; A third determination module is used to determine a target cabin whose activity reaches a preset activity threshold based on the activity change curve; A judgment module, used for judging whether there is an abnormal cabin in the target cabin based on the environmental quality change curve and the activity change curve; The control module is used to control the ventilation device to ventilate the abnormal cabin when there is an abnormal cabin.

9. An electronic device, characterized in that: It includes: at least one processor; Memory; At least one application, wherein the at least one application is stored in the memory and is configured to be executed by the at least one processor, and the at least one application is used to execute a monitoring method based on a multimodal laboratory environment according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed in a computer, the computer is caused to execute the multimodal laboratory environment monitoring method according to any one of claims 1 to 7.

Images