Abnormal state monitoring method and device for scene and readable storage medium

Abstract

The embodiment of the present disclosure relates to the field of smart home, and provides a scene abnormal state monitoring method and device and a readable storage medium. The method comprises the following steps: collecting first scene information of a target scene, wherein the first scene information comprises scene information of the target scene collected in a current time period; comparing the first scene information with reference scene information of the target scene to obtain an information difference value; determining that an abnormal state exists in the target scene when the information difference value is greater than or equal to a difference threshold; collecting second scene information related to the abnormal state, wherein the second scene information comprises scene information of the target scene collected in a first time period before the current time period; determining a current credibility of the abnormal state according to the first scene information and the second scene information; and processing the abnormal state when the current credibility is greater than a credibility threshold. The method provided by the present application can effectively improve the accuracy of detecting the abnormal state in the target scene.

Description

[0001] This application claims priority to patent application filed on August 13, 2026, with the Intellectual Property Office of Singapore, application number 10202602817T, entitled “METHOD AND APPARATUS FOR MONITORING ABNORMAL STATE OFSCENE, AND READABLE STORAGE MEDIUM”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of smart home technology, and in particular to a method, device and readable storage medium for monitoring abnormal states in a scenario. Background Technology

[0003] With the rapid popularization of smart home devices, during the use of smart home devices, abnormal states that occur in the home scene can be detected. When an abnormal state is detected, the smart home device can also issue an alarm message to notify the user.

[0004] However, in actual operation, smart home devices have a low accuracy rate in detecting abnormal states, and often produce false alarms and other issues, which affect the user experience. Summary of the Invention

[0005] The first aspect of this disclosure provides a method for monitoring abnormal states in a scenario, including: Collect the first scene information of the target scene, which includes the scene information of the target scene collected within the current time period; The first scene information is compared with the baseline scene information of the target scene to obtain the information difference value. The baseline scene information is the scene information that is stored in advance when the target scene is in a normal state. If the information difference value is greater than or equal to the difference threshold, it is determined that there is an abnormal state in the target scene; Collect second scene information related to abnormal states. The second scene information includes scene information of the target scene collected in the first time period before the current time period. The current credibility of the abnormal state is determined based on the information from the first scenario and the information from the second scenario. If the current credibility level is greater than the credibility threshold, handle the abnormal state.

[0006] A second aspect of this disclosure provides a device for monitoring abnormal states in a scene, comprising: The acquisition module is used to acquire first scene information and second scene information of the target scene. The first scene information includes the scene information of the target scene acquired within the current time period, and the second scene information includes the scene information of the target scene acquired within the first time period before the current time period. The memory module is used to store baseline scene information, which is pre-stored scene information corresponding to the target scene when it is in a normal state; The comparison module is used to compare the first scene information with the baseline scene information to obtain the information difference value; The first determining module is used to determine that there is an abnormal state in the target scene when the information difference value is greater than or equal to the difference threshold. The second determining module is used to determine the current credibility of the abnormal state based on the first scenario information and the second scenario information; The processing module is used to handle abnormal states when the current credibility level is greater than the credibility threshold.

[0007] A third aspect of this disclosure provides a computer-readable storage medium storing a computer program for executing the abnormal state monitoring method for the scenario described in the first aspect.

[0008] A fourth aspect of this disclosure provides an electronic device including a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the executable instructions to implement an abnormal state monitoring method for the scenario proposed in the first aspect.

[0009] The technical solution provided in this disclosure determines the state of the target scene based on the information difference between the first scene information and the baseline scene information. If an abnormal state is determined to exist in the target scene, second scene information related to the abnormal state is collected from the first time period before the current time period. Then, the current credibility of the abnormal state of the target scene is determined by combining the first scene information and the second scene information. Only when the current credibility is greater than the credibility threshold is the abnormal state of the target scene alarmed or intervened. This can effectively improve the accuracy of abnormal state detection in the target scene, reduce false alarms and other phenomena, and improve the user experience. Attached Figure Description

[0010] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is an architectural block diagram of an abnormal state monitoring system for a scenario provided in an exemplary embodiment of this disclosure; Figure 2 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in an exemplary embodiment of this disclosure; Figure 3 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in another exemplary embodiment of this disclosure; Figure 4 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in yet another exemplary embodiment of this disclosure; Figure 5 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in yet another exemplary embodiment of this disclosure; Figure 6 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in yet another exemplary embodiment of this disclosure; Figure 7 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in yet another exemplary embodiment of this disclosure; Figure 8 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in yet another exemplary embodiment of this disclosure; Figure 9 This is a schematic diagram of the structure of an abnormal state monitoring device for a scenario provided in an exemplary embodiment of this disclosure; Figure 10 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of the present disclosure. Detailed Implementation

[0012] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0013] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0014] Application Overview With the rapid popularization of smart home devices, home cameras, robot vacuums, smart door locks, smart speakers, PTZ devices, millimeter-wave radar, environmental sensors, smart lights, pet feeders, and other smart home devices have been widely deployed in home scenarios. During the use of smart home devices, the environmental sensors or cameras integrated into the smart home devices can be used to detect abnormal states in the home scenario. When an abnormal state is detected, the smart home devices can also issue alarm information to alert the user.

[0015] For example, when smart home devices detect phenomena such as smoke, water accumulation on the floor, or doors opening automatically in a home environment, they will issue corresponding alarm messages to alert the user that an abnormal state has occurred in the home environment.

[0016] However, in related technologies, smart home devices rely on sensors, cameras, and other components to collect data and images, and then use this data and images to determine abnormal states. In other words, smart home devices can only determine whether the current state of the scene is abnormal based on the current data and images, but cannot understand the formation process of the current state of the scene, which can lead to misjudgments of abnormal states of the scene.

[0017] For example, when a smart home device detects water on the floor, it determines that the current scene is in an abnormal state. However, there are many reasons why water might be on the floor, such as a tilted or fallen cup. The reasons for a tilted cup can also be varied, such as a person shaking the cup while holding it, a pet knocking it over while running, or a person intentionally tilting the cup while cleaning the floor, or a child intentionally tilting it while playing. Smart home devices can only detect the final result of water on the floor and issue an alarm, but they cannot understand the entire chain of events that caused the water to appear. This leads to false alarms and negative user experience in some cases.

[0018] To address the aforementioned technical issues, this disclosure provides a method for monitoring abnormal states in a scene, applicable to smart home devices such as home cameras, robot vacuums, smart door locks, smart speakers, PTZ devices, millimeter-wave radar, environmental sensors, smart lighting, and pet feeders. During the operation of these smart home devices, they can collect primary scene information of the target scene through integrated environmental sensors, cameras, and other components; that is, they can collect scene information of the target scene in real time within the current time period.

[0019] Then, the smart home device can further compare the collected first scene information with pre-stored baseline scene information that indicates the target scene is in a normal state, using a processor. This allows for the calculation of the information difference value between the first scene information and the baseline scene information. The information difference value quantifies the degree of deviation between the first scene information of the current target scene and the baseline scene information corresponding to the target scene in a normal state; a larger value indicates a more severe deviation from the normal state. It is understandable that since the baseline scene information indicates the target scene is in a normal state, if the information difference value between the first scene information and the baseline scene information is too large, it indicates that the current target scene is in an abnormal state. In other words, if the information difference value between the first scene information and the baseline scene information is greater than or equal to a difference threshold, it can be determined that an abnormal state exists in the current target scene.

[0020] If smart home devices detect an abnormal state in the current target scene, they can collect second scene information related to the abnormal state. It's important to note that the second scene information can include scene information collected within a first time period preceding the current time period. In other words, the first scene information collected within the current time period is used to analyze whether the target scene has an abnormal state within that time period. However, if an abnormal state is determined to exist within the current time period, the second scene information from the first time period preceding the current time period can be retrieved to analyze the cause of the abnormal state, thus avoiding misjudgments of the target scene's abnormal state based solely on the first scene information from the current time period.

[0021] It is understandable that the second scene information is scene information related to the detected abnormal state. Therefore, when collecting the second scene information, we can first identify the smart home devices related to the abnormal state, and then retrieve the data from the first time period in the smart home device's memory to realize the collection of the second scene information.

[0022] After collecting the second scene information, the current credibility of the abnormal state of the target scene can be determined based on the first and second scene information. In other words, based on the first scene information, it can be determined that the current target scene is in an abnormal state. Then, by combining the second scene information from the first time period prior to the current time period, the possible causes of the current abnormal state can be analyzed, thereby determining whether the abnormal state was caused by an accident, i.e., determining whether the abnormal state is credible. Only when the credibility of the abnormal state is greater than a credibility threshold can the currently detected abnormal state be determined to be credible and potentially have a negative impact on the user's normal life.

[0023] Finally, once the current credibility of the abnormal state is determined to be greater than the credibility threshold, the abnormal state can be processed. For example, the smart home devices that can issue alarm information can be controlled to issue corresponding alarm information, thereby prompting the user that there is an abnormal state in the current target scene and that it needs to be dealt with in a timely manner.

[0024] Thus, the abnormal state monitoring method for scenarios provided in this embodiment determines the state of the target scenario based on the information difference between the first scenario information and the baseline scenario information. When an abnormal state is determined to exist in the target scenario, second scenario information related to the abnormal state is collected from the target scenario within a first time period before the current time period. Then, the current credibility of the abnormal state of the target scenario is determined by combining the first scenario information and the second scenario information. Only when the current credibility is greater than the credibility threshold is the target scenario's current abnormal state alarmed or intervened. This can effectively improve the accuracy of abnormal state detection in the target scenario, reduce false alarms and other phenomena, and improve user experience.

[0025] Exemplary System Figure 1 This is an architectural block diagram of an abnormal state monitoring system for a scenario provided in an exemplary embodiment of this disclosure.

[0026] like Figure 1 As shown, the abnormal state monitoring system 10 for a scene may include a scene information acquisition module 11, a baseline scene information establishment module 12, an abnormal state analysis module 13, a credibility reasoning module 14, an abnormal impact assessment module 15, and an abnormal state processing module 16.

[0027] The scene information acquisition module 11 can include multiple smart home devices within the target scene, such as home cameras, robot vacuums, smart door locks, smart speakers, pan-tilt devices, millimeter-wave radar, environmental sensors, smart lights, pet feeders, etc. Through the environmental sensors or cameras integrated into the smart home devices, scene information of the home scene, i.e., the target scene, can be collected. Additionally, the smart home devices can also integrate a memory to store the collected scene information. This information can be retrieved later when an abnormal state is detected in the target scene, allowing for analysis of the credibility of the abnormal state.

[0028] The abnormal state analysis module 13 can analyze the first scene information collected by the scene information acquisition module 11 to determine whether an abnormal state exists in the target scene. For example, the abnormal state analysis module 13 can compare the first scene information with pre-stored baseline scene information that indicates the target scene is in a normal state, thereby calculating the information difference value between the first scene information and the baseline scene information. If the information difference value between the first scene information and the baseline scene information is greater than or equal to a difference threshold, it can be determined that an abnormal state exists in the current target scene.

[0029] The credibility reasoning module 14 can determine the current credibility of the abnormal state of the target scene based on the first scene information and the second scene information. The second scene information is collected by the scene information acquisition module 11 within a first time period prior to the current time period and stored in its integrated memory. Based on the first scene information, it can be determined that the current target scene is in an abnormal state. Then, by combining the second scene information from the first time period prior to the current time period, the possible causes of the current abnormal state are analyzed, thereby determining whether the abnormal state is due to an accident, that is, determining whether the abnormal state is credible. Only when the credibility of the abnormal state is greater than a credibility threshold can the currently detected abnormal state be determined to be credible, potentially having a negative impact on the user's normal life.

[0030] The anomaly impact assessment module 15 can assess the potential impact of the currently detected anomaly, that is, assess the degree of impact of the current anomaly on the user's daily life. For example, when an anomaly of water on the ground is detected in the target scene, the scene information acquisition module 11 can collect scene information in the current target scene, and then the anomaly impact assessment module 15 can determine whether the water on the ground is near a socket, whether it is near a robot charging base, whether it may affect the operation of other equipment, whether it may cause children to slip, etc.

[0031] The abnormal state handling module 16 can process an abnormal state when its current credibility exceeds a credibility threshold. For example, the abnormal state handling module 16 can also include smart home devices operating within the target scene. Alarm information corresponding to the abnormal state can be issued through the interactive unit integrated into the smart home device to prompt the user to handle the abnormal state promptly. Alternatively, the abnormal state handling module 16 can directly intervene in the abnormal state to resolve it or prevent its further spread. For example, when an abnormal state of water on the ground is detected, a robot vacuum cleaner can directly handle the water on the ground to prevent it from spreading.

[0032] The abnormal state monitoring system 10 of this embodiment, through the coordinated operation of a scene information acquisition module 11, a baseline scene information establishment module 12, an abnormal state analysis module 13, a credibility reasoning module 14, an abnormal impact assessment module 15, and an abnormal state processing module 16, achieves the determination of the current credibility of the abnormal state in the target scene by combining first scene information and second scene information when an abnormal state is determined to exist in the target scene. This effectively improves the accuracy of abnormal state detection in the target scene, reduces false alarms and other phenomena, and improves user experience. Furthermore, it can also assess the degree of impact of the current abnormal state on the user's daily life and take proactive intervention to resolve the abnormal state or prevent its further spread.

[0033] Exemplary methods Figure 2 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in an exemplary embodiment of this disclosure. This embodiment can be applied to smart home devices, such as... Figure 2 As shown, the method includes the following steps: Step S100: Collect the first scene information of the target scene.

[0034] The first scene information includes scene information of the target scene collected within the current time period.

[0035] In one possible approach, during the operation of smart home devices, the first scene information of the target scene can be collected through integrated environmental sensors, cameras, and other components, that is, the scene information of the target scene within the current time period can be collected in real time.

[0036] By adopting this approach, real-time data collection of the first scene information of the target scenario can be achieved through smart home devices, thereby providing a basis for determining whether there are any abnormal states in the target scene.

[0037] Step S200: Compare the first scene information with the baseline scene information of the target scene to obtain the information difference value.

[0038] Among them, the baseline scene information is the scene information that is pre-stored when the target scene is in a normal state.

[0039] In one possible implementation, the smart home device can also use a processor to compare the collected first scene information with pre-stored reference scene information that indicates the target scene is in a normal state, thereby calculating the information difference value between the first scene information and the reference scene information.

[0040] It is understandable that, since the baseline scene information is used to indicate the scene information when the target scene is in a normal state, by comparing the information difference value between the first scene information and the baseline scene information, it is possible to determine whether the current target scene is in an abnormal state.

[0041] For example, to calculate the information difference value, the multi-dimensional feature vector of the current scene information collected in real time can be quantitatively compared with the standard feature vector of the pre-constructed benchmark scene information to obtain a comprehensive difference index. For example, methods such as image cosine similarity difference, Euclidean distance normalization of multi-sensor data, and mean normalization of map coordinate offset in Simultaneous Localization and Mapping (SLAM) can be used to calculate the differences between feature vectors of different dimensions. Then, through weighted fusion, the differences of multiple dimensions are fused to obtain the final information difference value. This information difference value can be used to quantify the degree of deviation between the first scene information of the current target scene and the benchmark scene information corresponding to the target scene under normal conditions. The larger the value, the more serious the deviation of the scene from the normal state. Multi-dimensional features can include: image visual features, such as object coordinates, contours, and average pixel values ​​of regions; environmental sensor data, such as humidity, water accumulation, smoke, and radar signals of humans or pets; device status data, such as the open / closed status of doors and windows, the on / off status of stoves, and the status of lights; and spatial location SLAM point information, etc.

[0042] Step S300: If the information difference value is greater than or equal to the difference threshold, determine that there is an abnormal state in the target scene.

[0043] In one possible implementation, if the information difference between the first scene information and the baseline scene information is greater than or equal to the difference threshold, it can be determined that there is an abnormal state in the current target scene.

[0044] By adopting the implementation method of this step, the information difference value between the first scene information and the baseline scene information is compared with the difference threshold, thereby providing a quantifiable basis for judging whether there is an abnormal state in the target scene, which can effectively simplify the judgment logic and improve the judgment efficiency.

[0045] Step S400: Collect second scenario information related to the abnormal state.

[0046] The second scene information includes scene information of the target scene collected in the first time period before the current time period.

[0047] In one possible implementation, if a smart home device detects an abnormal state in the current target scene, it can collect second scene information related to the abnormal state.

[0048] It should be noted that the second scene information can include scene information of the target scene collected in the first time period prior to the current time period. In other words, the first scene information collected in the current time period is used to analyze whether the target scene has an abnormal state in the current time period. However, if an abnormal state is determined to exist in the current time period, the second scene information from the first time period prior to the current time period can be retrieved to analyze the cause of the abnormal state, thus avoiding misjudgment of the abnormal state of the target scene due to relying solely on the first scene information of the current time period.

[0049] It is understandable that the second scene information is scene information related to the detected abnormal state. Therefore, when collecting the second scene information, we can first identify the smart home devices related to the abnormal state, and then retrieve the data from the first time period in the smart home device's memory to realize the collection of the second scene information.

[0050] Step S500: Determine the current credibility of the abnormal state based on the first scenario information and the second scenario information.

[0051] In one possible approach, after collecting the second scene information, the current credibility of the abnormal state of the target scene can be determined based on the first and second scene information.

[0052] In other words, based on the first scene information, it can be determined that the current target scene is in an abnormal state. Then, by combining the second scene information from the first time period before the current time period, the possible causes of the current abnormal state can be analyzed, and it can be determined whether the abnormal state is caused by an accident, that is, whether the abnormal state is credible. Only when the credibility of the abnormal state is greater than the credibility threshold can it be determined that the currently detected abnormal state is credible and may have a negative impact on the user's normal life.

[0053] Step S600: If the current credibility is greater than the credibility threshold, handle the abnormal state.

[0054] In one possible approach, once the current credibility of the abnormal state is determined to be greater than a credibility threshold, the abnormal state can be addressed. For example, smart home devices capable of issuing alarms can be controlled to send corresponding alarm messages, thereby alerting the user that an abnormal state exists in the current target scene and requires timely intervention. Alternatively, smart home devices integrated with environmental sensors or image acquisition devices can be controlled to continuously collect scene information related to the abnormal state, thereby continuously observing the changing trend of the abnormal state. Or, the operation of smart home devices capable of altering the abnormal state can be controlled to address the abnormal state, thereby resolving it or reducing its spread.

[0055] The abnormal state monitoring method of this disclosure determines the state of the target scene based on the information difference between the first scene information and the baseline scene information. When an abnormal state is determined to exist in the target scene, second scene information related to the abnormal state is collected from the target scene in a first time period before the current time period. Then, the current credibility of the abnormal state of the target scene is determined by combining the first scene information and the second scene information. Only when the current credibility is greater than the credibility threshold is the target scene's current abnormal state alarmed or intervened. This can effectively improve the accuracy of abnormal state detection in the target scene, reduce false alarms and other phenomena, and improve user experience.

[0056] Figure 3 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in another exemplary embodiment of this disclosure.

[0057] like Figure 3 As shown, in one implementation, step S500: determining the current credibility of the abnormal state based on the first scenario information and the second scenario information may include the following steps: Step S510: Determine the state change chain of the abnormal state based on the first scenario information and the second scenario information.

[0058] Among them, the state change chain is used to characterize the formation process of abnormal states.

[0059] In one possible implementation, the second scene information may include scene information of the target scene collected within a first time period prior to the current time period. Thus, when an abnormal state is determined to exist in the current target scene based on the first scene information, the second scene information collected within the first time period can be used to determine how the abnormal state was formed, that is, to identify the state change chain of the abnormal state, thereby characterizing the formation process of the abnormal state through the state change chain.

[0060] For example, when the first scene information determines that there is an abnormal state of water on the ground in the current target scene, the second scene information within the first time period before the current time can be collected. For example, the second scene information may include video information of the water-covered ground within the first time period, so as to determine whether the water was caused by the user deliberately pouring it or by the user or pet accidentally knocking over the water cup.

[0061] Step S520: Obtain the first sharpness of the image information in the first scene information and the second scene information.

[0062] In one possible implementation, if image information is included in the first scene information and the second scene information, the first sharpness of the image information can also be obtained.

[0063] Understandably, when judging the accuracy of an abnormal state using image information, the clarity of the image information is a crucial factor in determining the accuracy of the judgment. For example, factors affecting the clarity of image information may include ground reflection, shadow coverage, object obstruction, or low light conditions at night.

[0064] Step S530: Determine the first reasonableness of the formation process of the abnormal state based on the state change chain.

[0065] In one feasible approach, after determining the state change chain of an abnormal state, the first rationality of the formation process of the abnormal state represented by the state change chain can be analyzed based on the state change chain.

[0066] Understandably, the primary rationality of the formation process of an abnormal state refers to whether the formation process conforms to basic laws, such as the normal behavioral patterns of people or pets, or the physical laws governing objects. In other words, when the rationality of the formation process of an abnormal state is high, the credibility of the current abnormal state can be judged to be high; conversely, if the rationality of the formation process is low, the credibility of the abnormal state is also relatively low.

[0067] For example, the calculation of the first reasonableness can be performed by breaking down each logical behavior in the state change chain of the abnormal state and matching it with the pre-stored home behavior and physical common sense rules in the normal state. If a compliant logical behavior is successfully matched, the corresponding score is increased; if it violates common sense or has a logical gap, the score is deducted. After all the matching is completed, the score corresponding to each logical behavior in the state change chain is normalized and the first reasonableness is output.

[0068] Step S540: Determine the first accuracy of the first scene information and the second scene information based on the first resolution.

[0069] In one possible implementation, since the clarity of image information is an important factor in determining the accuracy of abnormal state judgment, when the first clarity is low, it means that the accuracy of the first and second scene information is poor, and thus it can be determined that there is a high possibility of misjudgment of the current abnormal state. Conversely, if the first clarity of image information is high, it means that the accuracy of the first and second scene information is high.

[0070] In other words, after obtaining the first clarity of the first scene information and the second scene information, the first accuracy of the first scene information and the second scene information can be judged based on the first clarity, and then the first accuracy can be used as an important basis for judging the current credibility of the abnormal state.

[0071] For example, the calculation of the first accuracy can be performed using the image gradient variance method to calculate the first sharpness score. That is, the grayscale gradient variance can be calculated for the effective detection area of ​​the image information. The larger the gradient variance, the sharper the image edges and object outlines, and the higher the first sharpness score. Reflections, low light, occlusion, and motion blur significantly reduce the gradient variance, resulting in a lower first sharpness score. Then, the range of the first sharpness score is mapped to the accuracy score range of 0 to 1, and the actual first sharpness score obtained from the image information is normalized to obtain the first accuracy corresponding to the first scene information and the second scene information. It can be understood that the higher the first sharpness score, the higher the corresponding first accuracy score.

[0072] Step S550: Determine the current credibility based on the first reasonableness and the first accuracy.

[0073] In one feasible approach, after obtaining the first reasonableness of the formation process of the abnormal state and the first accuracy of the first scenario information and the second scenario information, the first reasonableness and the first accuracy can be combined. By comprehensively considering the formation process of the abnormal state and the accuracy of the collected scenario information, the current credibility of the abnormal state can be judged, thereby effectively improving the accuracy of the abnormal state judgment.

[0074] For example, weights can be set for the first reasonableness and the first accuracy first, and then a weighted fusion calculation can be performed on the first reasonableness and the first accuracy to obtain the current credibility. For example, the weight of the first reasonableness can be set to 0.6, and the weight of the first accuracy can be set to 0.4. That is, by setting the weight of the first reasonableness higher, the cause of the abnormal state is judged first, and then the credibility of the abnormal state is judged in combination with the image quality.

[0075] The technical solution of this disclosure, when an abnormal state is detected in a target scene, firstly, determines a state change chain of the abnormal state based on first scene information and second scene information to characterize the formation process of the abnormal state, and determines the first reasonableness of the formation process of the abnormal state based on the state change chain; then, determines the first accuracy of the first scene information and second scene information based on the first clarity of the image information in the first scene information and second scene information; finally, combines the first reasonableness and the first accuracy, comprehensively considers the formation process of the abnormal state and the accuracy of the collected scene information, and judges the current credibility of the abnormal state, thereby effectively improving the accuracy of abnormal state judgment, avoiding false alarms and misreporting of abnormal states, and improving user experience.

[0076] Figure 4 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in yet another exemplary embodiment of this disclosure.

[0077] like Figure 4 As shown, in one implementation, after determining the current credibility of the abnormal state based on the first scenario information and the second scenario information, step S500 may further include the following steps: Step S700: If the current confidence level is less than or equal to the confidence level threshold, collect supplementary scenario information related to the abnormal state.

[0078] The supplementary scene information includes scene information of the target scene collected after the current time period. The collection strategy for the supplementary scene information is different from that for the first scene information.

[0079] In one possible approach, if the current credibility of the abnormal state is less than or equal to the credibility threshold, it indicates that the result of analyzing the target scene based on the first scene information may be a misjudgment. In this case, to avoid missing the abnormal state, supplementary scene information related to the abnormal state can be collected through smart home devices, and then the abnormal state can be further confirmed based on the supplementary scene information to further improve the accuracy of the abnormal state judgment.

[0080] It is understandable that supplementary scene information can be scene information of the target scene collected after the current time period. In other words, if the current credibility of the abnormal state obtained based on the scene information collected in the first time period before the current time period is insufficient, scene information related to the abnormal state can be collected in subsequent time periods to further confirm the abnormal state.

[0081] Furthermore, the strategy for collecting supplementary scene information differs from the strategy for collecting the first scene information. In other words, if the current credibility of the abnormal state determined based on the first scene information is insufficient, the collection strategy can be changed to collect scene information related to the abnormal state again, further confirming the abnormal state. This allows for judging the credibility of the abnormal state from different perspectives, further improving the accuracy of the abnormal state judgment. For example, the collection strategy may include the type of collection device, the operating parameters of the collection device, and the location of the collection device. That is, when the current credibility is insufficient, the collection strategy can be changed by switching to other collection devices, changing the operating parameters of the current collection device, or changing the location of the collection device, thereby enabling the collection of supplementary scene information.

[0082] Step S800: Update the current credibility of the abnormal state based on the supplementary scenario information.

[0083] In one possible implementation, after acquiring supplementary scenario information related to the abnormal state, the current credibility of the abnormal state can be updated based on this supplementary scenario. That is, when the credibility of the current abnormal state of the target scenario is insufficient based on the first and second scenario information, supplementary scenario information related to the abnormal state is continuously collected, and the current credibility of the abnormal state is further judged based on this supplementary scenario information, thereby updating the current credibility of the abnormal state. Then, based on the updated current credibility, the abnormal state is further confirmed.

[0084] It is understandable that, since the supplementary scene information is obtained after the current time period, and the collection strategy for the supplementary scene information is different from that for the first scene information, the current credibility of the abnormal state can be judged based on different observation times and different observation angles, thereby improving the accuracy of abnormal state judgment.

[0085] Step S900: If the updated current credibility is greater than the credibility threshold, handle the abnormal state.

[0086] In one possible approach, if the updated current credibility is greater than the credibility threshold, the abnormal state can be handled. For example, the smart home device that can issue alarm information can be controlled to issue the corresponding alarm information, thereby prompting the user that there is an abnormal state in the current target scene and that it needs to be handled in time.

[0087] Step S1000: If the updated current credibility is less than or equal to the credibility threshold, proceed to step S700.

[0088] In one possible approach, if the updated current credibility is still less than or equal to the credibility threshold, it indicates that the credibility of the result of the target scenario being in an abnormal state based on the supplementary scenario information is still insufficient and there is still a possibility of misjudgment. The abnormal state can be further analyzed and confirmed to avoid missing the abnormal state and causing accidents.

[0089] In other words, if the updated current credibility is less than or equal to the credibility threshold, the process can jump to step S700 above and continue to execute steps S800 and S900 or S1000 until it is confirmed that the current credibility of the abnormal state is greater than the credibility threshold, or until it is confirmed that the abnormal state has been resolved.

[0090] The technical solution of this disclosure, when determining that the current credibility of an abnormal state is less than or equal to a credibility threshold, continues to collect supplementary scene information related to the abnormal state in a subsequent time period based on a collection strategy different from the collection strategy for the first scene information. This allows for updating the current credibility of the abnormal state based on the supplementary scene information, thereby further confirming the abnormal state and effectively improving the accuracy of abnormal state judgment. Furthermore, when the updated current credibility is less than the credibility threshold, the process of collecting supplementary scene information and updating the current credibility can be repeated cyclically, effectively avoiding missed detections of abnormal states and preventing unexpected consequences.

[0091] Figure 5 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in yet another exemplary embodiment of this disclosure.

[0092] like Figure 5 As shown, in one implementation, step S800: updating the current confidence level of the abnormal state based on supplementary scenario information may include the following steps: Step S810: Update the state change chain of the abnormal state based on the supplementary scenario information.

[0093] In one possible approach, after acquiring supplementary scene information related to the abnormal state, the supplementary scene information can be combined with the previously acquired first and second scene information. This combines the scene changes before and after the detection of the abnormal state to infer the state change chain of the abnormal state, thereby updating the state change chain of the abnormal state. The updated state change chain is then used to characterize the formation process of the abnormal state.

[0094] Step S820: Obtain the second sharpness of the image information in the supplementary scene information.

[0095] In one possible approach, when image information is included in the supplementary scene information, a second level of image clarity can also be obtained.

[0096] Similar to the reason for obtaining the first level of clarity, when the supplementary scene information includes image information, the clarity of the image information is an important factor in determining the accuracy of the judgment. Therefore, the second level of clarity of the image information in the supplementary scene information can be obtained to provide a basis for judging the current credibility of the abnormal state in subsequent updates.

[0097] Step S830: Determine the second plausibility of the formation process of the abnormal state based on the updated state change chain.

[0098] In one feasible approach, after updating the state change chain of the anomalous state, the second rationality of the formation process of the anomalous state represented by the updated state change chain can be analyzed.

[0099] Step S840: Determine the second accuracy of the supplementary scene information based on the second clarity.

[0100] In one possible implementation, after obtaining the second clarity of the image information of the supplementary scene information, the second accuracy of the supplementary scene information can be determined based on the second clarity, and then the second accuracy can be used as an important basis for the current credibility of the subsequent update of the abnormal state.

[0101] Step S850: Determine the current credibility based on the second reasonableness and the second accuracy.

[0102] In one possible approach, after obtaining the second rationality of the formation process of the abnormal state and the second accuracy of the supplementary scenario information, the second rationality and the second accuracy can be combined. By comprehensively considering the formation process of the abnormal state and the accuracy of the further collected supplementary scenario information, the current credibility of the abnormal state can be updated, thereby effectively improving the accuracy of the abnormal state judgment.

[0103] The technical solution of this disclosure, when determining that the current credibility of an abnormal state is less than or equal to a credibility threshold, firstly updates the state change chain of the abnormal state based on the collected supplementary scene information, and then updates the second rationality of the formation process of the abnormal state based on the updated state change chain; then, determines the second accuracy of the supplementary scene information based on the second clarity of the image information in the supplementary scene information; finally, combines the second rationality and the second accuracy, and comprehensively considers the scene changes before and after the abnormal state, updates the current credibility of the abnormal state, thereby effectively improving the accuracy of abnormal state judgment, avoiding false alarms and misreports of abnormal states, and improving user experience.

[0104] Figure 6 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in yet another exemplary embodiment of this disclosure.

[0105] like Figure 6 As shown, in one implementation, step S600: when the current credibility is greater than the credibility threshold, handling the abnormal state may include the following steps: Step S610: Analyze the state change chain of the abnormal state to obtain the abnormal impact information of the abnormal state.

[0106] Among them, the abnormal impact information refers to the impact range and abnormal results of the abnormal state obtained by predicting the abnormal state.

[0107] In one feasible approach, the abnormal state can be processed once the current credibility of the abnormal state is determined to be greater than the credibility threshold. In the process of processing the abnormal state, the state change chain of the abnormal state can be analyzed first to obtain the abnormal impact information of the abnormal state.

[0108] For example, a pre-trained neural network model can be used to process the state change chain of an abnormal state, thereby predicting the abnormal state, that is, predicting the impact and abnormal results of the abnormal state in subsequent time periods, thus obtaining information on the abnormal impact of the abnormal state. The neural network model can include a Vision Language Model (VLM), a Large Language Model (LLM), etc. For example, for the abnormal state of water on the ground, scene information related to this abnormal state can be input into the neural network model. The neural network model can predict the range of influence that the water on the ground can reach based on the amount of water accumulation, its spread trend, spread direction, and spread speed. For example, it can predict whether the water will be near a socket or a robot charging dock; it can also predict whether the water will affect the operation of other equipment or cause children to slip and fall, etc.

[0109] Step S620: Determine the anomaly level of the anomaly state based on the anomaly impact information.

[0110] In one feasible approach, the anomalous level of an anomalous state can be determined based on its impact range and consequences. Understandably, the anomalous level characterizes the severity of the potential consequences of the anomalous state. A higher anomalous level indicates more severe consequences, requiring timely intervention; conversely, a lower anomalous level indicates that the anomalous state will not cause particularly serious consequences.

[0111] For example, the anomaly level can be divided into three levels: low, medium, and high. The higher the level, the more serious the potential hazard, loss, or safety risk. The scope of impact can include three situations: slight local spread, local spread, and large-scale spread throughout the house. The abnormal outcome can include various situations such as minor damage to items, slips and falls, electrical leaks and fires, and flooding damaging appliances. When determining the anomaly level, each situation in the scope of impact and abnormal outcome can be assigned a score from 0 to 5. The more serious the safety risk or the larger the scope of impact, the higher the corresponding score. Then, a weighted summation is used to obtain the total score of the current abnormal state. Finally, the anomaly level of the abnormal state is determined based on this total score. For example, a total score in the range of 0 to 2 indicates a low risk level, a total score in the range of 2 to 4 indicates a medium risk level, and a total score in the range of 4 to 5 indicates a high risk level.

[0112] Step S630: Based on the abnormal impact information and abnormal level of the abnormal state, generate a handling plan for the abnormal state.

[0113] The treatment plan includes at least one of the following: a prompting plan, an observation plan, and an intervention plan.

[0114] In one feasible approach, after determining the anomaly level of an abnormal state, the anomaly impact information of the abnormal state can be combined with the anomaly level to comprehensively consider the scope of the anomaly's impact, the abnormal results, and the severity of the possible consequences, thereby generating a corresponding handling plan to ensure the rationality of the handling process for the abnormal state.

[0115] For example, the handling scheme for abnormal states may include a notification scheme. That is, the user can be notified of the existence of an abnormal state in the current target scene by sending a notification message. This notification message can be a voice broadcast. Through voice broadcasting, not only can the user be notified of the abnormal state, but the cause, scope of impact, and abnormal result of the abnormal state can also be directly announced, thereby facilitating the user's handling of the abnormal state.

[0116] The handling plan for abnormal states can also include an observation plan. That is, if the current abnormal state has a minor impact on the target scenario and will not cause serious consequences, the abnormal state can be observed to analyze its changing trends, determine whether it is escalating, and whether it may cause serious consequences. Only when necessary should a warning message be issued or proactive intervention be taken to handle the current abnormal state without affecting the user's normal daily life.

[0117] Solutions for handling abnormal states can also include intervention measures. Intervention measures involve controlling smart home devices in the target scenario to resolve the abnormal state or reduce its spread. For example, for the abnormal state of water on the floor, a robot vacuum cleaner can be used to remove the water, thus resolving the abnormal state. Alternatively, if a cookware has been heating for too long, the smart stove can be stopped to prevent food from burning or causing a fire.

[0118] Understandably, since the anomaly impact information is the predicted scope and outcome of the abnormal state, meaning the anomaly level is determined by combining both, there might be cases where the anomaly level is low, but one of the impact scope and outcome scores is high while the other is low. Therefore, it's necessary to combine the anomaly level with the anomaly impact information to determine the appropriate handling plan to ensure its rationality. For example, if the anomaly level is low and the impact scope is small, unlikely to cause a safety incident, an observation plan can be chosen. Conversely, even if the anomaly level is low but the impact scope is large and unlikely to cause a safety incident, a combination of observation and alert plans can be used. Alternatively, even if the anomaly level is low and the impact scope is small but could potentially cause a safety incident, a combination of alert and intervention plans can be used.

[0119] Step S640: Handle the abnormal state according to the processing plan.

[0120] In one possible approach, once the handling scheme corresponding to the abnormal state is determined, the abnormal state can be handled according to the handling scheme.

[0121] Understandably, handling abnormal states can involve controlling the operation of smart home devices related to the abnormal state. For example, for a notification solution, smart home devices with voice broadcast or display functions can be controlled to broadcast and display notification information; for an observation solution, smart home devices integrated with environmental sensors or image acquisition devices can be controlled to continuously collect scene information related to the abnormal state, thereby continuously observing the changing trend of the abnormal state; for an intervention solution, the operation of smart home devices capable of changing the abnormal state can be controlled to handle the abnormal state, thereby resolving the abnormal state or reducing its spread.

[0122] The technical solution of this disclosure analyzes the abnormal state change chain to obtain abnormal impact information, further determines the abnormal level of the abnormal state based on the impact information, and then generates a processing scheme for handling the abnormal state based on the abnormal impact information and the abnormal level, so as to ensure the rationality of the abnormal state handling process.

[0123] Figure 7 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in yet another exemplary embodiment of this disclosure.

[0124] like Figure 7 As shown, in one implementation, step S640: processing the abnormal state according to the processing plan may include the following steps: Step S641: In response to the processing plan including the observation plan, determine the acquisition strategy corresponding to the observation plan.

[0125] In one feasible approach, when the current abnormal state has a minor impact on the target scenario and will not cause serious consequences, the abnormal state can be observed and its changing trend analyzed.

[0126] In other words, if the current handling solution for the abnormal state is observation, the first step is to determine the data acquisition strategy for that observation strategy. Understandably, for the observation strategy, it's typically necessary to control smart home devices integrated with environmental sensors or image acquisition equipment to continuously collect scene information related to the abnormal state, and then continuously observe the changing trends of the abnormal state. Therefore, before collecting scene information related to the abnormal state, the acquisition strategy related to the abnormal state can first be determined. This strategy can include the type of acquisition device, the operating parameters of the acquisition device, and the location of the acquisition device. Then, by switching to other acquisition devices, changing the operating parameters of the current acquisition device, or changing the location of the acquisition device, the acquisition strategy can be changed, thereby achieving the acquisition of third-party scene information related to the abnormal state.

[0127] Step S642: Collect third-scenario information related to the abnormal state according to the collection strategy.

[0128] In one possible approach, once the data collection strategy is determined, the operation of corresponding smart home devices can be controlled based on the strategy to collect third-scene information related to abnormal states within the target scene.

[0129] It is understandable that the third scenario information is collected after the abnormal state is detected, that is, the third scenario information is collected after the first scenario information. Furthermore, the collection strategy of the third scenario information can be the same as or different from the collection strategy of the first scenario information. Therefore, based on the third scenario information, the development trend of the abnormal state can be observed, which can prepare for subsequent handling of the abnormal state.

[0130] Step S643: Update the state change chain of the abnormal state based on the third scenario information.

[0131] In one possible approach, after collecting third scene information related to the abnormal state, the third scene information can be combined with the previously acquired first scene information and second scene information. This combines the scene changes before and after the detection of the abnormal state to infer the state change chain of the abnormal state, thereby updating the state change chain of the abnormal state. The updated state change chain can characterize the development trend of the abnormal state, and the handling scheme for the abnormal state can be updated based on the development trend.

[0132] Step S644: Update the abnormal impact information of the abnormal state according to the updated state change chain.

[0133] In one possible implementation, after the state change chain of the abnormal state is updated, the updated state change chain can be analyzed to update the abnormal impact information of the abnormal state.

[0134] For example, a pre-trained neural network model can be used to process the state change chain after the abnormal state is updated, thereby predicting the abnormal state, that is, predicting the impact and abnormal results of the abnormal state in subsequent time periods, thus realizing the update of the abnormal impact information of the abnormal state.

[0135] Step S645: Update the anomaly level of the anomaly state based on the updated anomaly impact information.

[0136] In one possible implementation, the anomalous level of an abnormal state can be updated based on the updated anomaly impact information. Thus, based on the updated anomalous level, the changes in the potential consequences of the abnormal state can be characterized, that is, it can be determined whether the abnormal state will cause serious consequences in the future.

[0137] Step S646: Update the handling plan based on the updated anomaly impact information and the updated anomaly level.

[0138] In one possible approach, after updating the anomaly level of an abnormal state, the updated anomaly impact information of the abnormal state can be combined with the updated anomaly level to comprehensively consider the scope of the anomaly state's impact, the anomaly result, and the severity of the possible consequences, thereby generating a corresponding handling plan to ensure the rationality of the handling process for the abnormal state.

[0139] Step S647: Handle the abnormal state based on the updated processing scheme.

[0140] In one possible implementation, after updating the handling scheme corresponding to the abnormal state, the abnormal state can be handled according to the updated handling scheme.

[0141] Understandably, the updated treatment plan can also include prompting, observation, or intervention plans.

[0142] The technical solution of this disclosure embodiment allows for handling of abnormal states based on an observation plan when the current abnormal state has a minor impact on the target scene and will not cause serious consequences. During the observation and handling process, third-scene information related to the abnormal state is collected. The state change chain, abnormal impact information, and abnormal level of the abnormal state are updated based on this third-scene information. Then, the handling plan is updated based on the updated abnormal impact information and the updated abnormal level. Finally, the abnormal state is handled according to the updated handling plan, ensuring the rationality of the abnormal state handling and improving the user experience.

[0143] Figure 8 This is a flowchart illustrating an abnormal state monitoring method for a scenario provided in yet another exemplary embodiment of this disclosure.

[0144] like Figure 8 As shown, in one implementation, before step S100: acquiring the first scene information of the target scene, the following steps may be included: Step S1100: During the second time period, collect historical scene information within the target scene.

[0145] The second time period is the period preceding the first time period, and the duration of the second time period is longer than that of the first time period.

[0146] In one possible approach, before determining whether there is an abnormal state in the target scene based on the first scene information, it is first necessary to establish the baseline scene information corresponding to the target scene. In this way, the first scene information can be compared with the baseline scene information of the target scene to obtain the information difference value, and then the abnormal state in the target scene can be detected based on the information difference value.

[0147] In establishing baseline scenario information, firstly, historical scenario information of the target scenario can be collected within a second time period. It should be noted that during the determination of the current credibility of the abnormal state, second scenario information from the first time period preceding the current time period is collected. The purpose of this second scenario information is to analyze the current credibility of the abnormal state; therefore, the first time period can only include a short segment of time preceding the current time period, such as 10 or 20 minutes. The purpose of historical scenario information is to establish baseline scenario information. Historical scenario information needs to include scenario information of the target scenario over a longer historical period. That is, the duration of the second time period will be longer than the duration of the first time period. For example, the second time period can be set to 7 days, 15 days, or one month.

[0148] For example, historical scene information can be regular color images, depth images, and SLAM images of the target scene. These images can reflect the home layout, the location of commonly used items, pet activity patterns, item information, space usage patterns, and the relative positional relationships between items within the target scene.

[0149] Step S1200: Determine the scene patterns of the target scene based on historical scene information.

[0150] Among them, scene patterns include at least one of the following: spatial layout patterns of the target scene, object distribution patterns, object activity patterns, and object association patterns.

[0151] In one feasible approach, the scene patterns of the target scene can be determined based on the acquired historical scene information.

[0152] For example, scene patterns may include the positions of sofas, dining tables, and cat trees in the target scene; the common placement of items such as remote controls, keys, and medicine boxes; the probability of pets moving around in different areas; the opening and closing patterns of items such as doors, children's fences, and kitchen lights; and the main uses of spaces such as dining tables, balconies, and children's rooms.

[0153] When determining the patterns of a target scene, corresponding neural network models can be used to process historical scene information. For example, a large visual language model can be used to process images of the target scene to obtain spatial layout patterns, object distribution patterns, and object activity patterns. The large language model can also be used to establish object association patterns between items.

[0154] Step S1300: Construct baseline scene information based on scene patterns.

[0155] In one feasible approach, baseline scene information corresponding to the target scene can be constructed based on the scene patterns of the target scene. Thus, when monitoring abnormal states of the target scene, the acquired first scene information can be compared with the baseline scene information. The information difference value between the first scene information and the baseline scene information can then be obtained based on the comparison result, and the existence of abnormal states in the target scene can be determined based on the information difference value.

[0156] The technical solution of this disclosure embodiment collects historical scene information of the target scene during the second time period, thereby analyzing the historical scene information to obtain the scene patterns of the historical scene. Then, based on the scene patterns, a benchmark scene information corresponding to the target scene is established. In this way, a judgment basis can be provided for subsequent monitoring of the abnormal state of the target scene based on the first scene information, thereby realizing the automated detection of abnormal states.

[0157] Exemplary device The above describes the abnormal state monitoring method for a scenario provided by the embodiments of this disclosure. It is understood that the abnormal state monitoring device for a scenario may include corresponding hardware and software for implementing the hardware functions in order to achieve the various functions of the abnormal state monitoring method for that scenario.

[0158] Those skilled in the art will readily recognize that the steps of the abnormal state monitoring method for a scenario described in conjunction with the embodiments of this disclosure can be implemented in hardware or in a software-driven hardware manner. Whether a function is executed in hardware or software-driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0159] The beneficial technical effects corresponding to the exemplary embodiments of this device can be found in the corresponding beneficial technical effects of the exemplary method section above, and will not be repeated here.

[0160] Figure 9 This is a schematic diagram of the structure of an abnormal state monitoring device for a scenario provided by an exemplary embodiment of this disclosure.

[0161] like Figure 9As shown, in one embodiment, the abnormal state monitoring device 900 for the scene includes: a collection module 910, used to collect first scene information and second scene information of the target scene, wherein the first scene information includes scene information of the target scene collected within the current time period, and the second scene information includes scene information of the target scene collected within a first time period before the current time period; a memory module 920, used to store baseline scene information, wherein the baseline scene information is pre-stored scene information corresponding to the target scene when it is in a normal state; a comparison module 930, used to compare the first scene information with the baseline scene information to obtain an information difference value; a first determination module 940, used to determine that there is an abnormal state in the target scene when the information difference value is greater than or equal to a difference threshold; a second determination module 950, used to determine the current credibility of the abnormal state based on the first scene information and the second scene information; and a processing module 960, used to process the abnormal state when the current credibility is greater than a credibility threshold.

[0162] The abnormal state monitoring device provided in this disclosure determines the state of the target scene based on the information difference between the first scene information and the baseline scene information. When an abnormal state is determined to exist in the target scene, it collects second scene information related to the abnormal state in the first time period before the current time period. Then, it combines the first scene information and the second scene information to determine the current credibility of the abnormal state of the target scene. Only when the current credibility is greater than the credibility threshold is the target scene's current abnormal state alarmed or intervened. This can effectively improve the accuracy of abnormal state detection in the target scene, reduce false alarms and other phenomena, and improve the user experience.

[0163] In one possible implementation, the second determining module 950 is configured to: determine a state change chain of an abnormal state based on first scene information and second scene information, the state change chain being used to characterize the formation process of the abnormal state; acquire a first clarity of image information in the first scene information and second scene information; determine a first reasonableness of the formation process of the abnormal state based on the state change chain; determine a first accuracy of the first scene information and second scene information based on the first clarity; and determine the current credibility based on the first reasonableness and the first accuracy.

[0164] In one possible implementation, the acquisition module 910 is further configured to: acquire supplementary scene information related to the abnormal state when the current credibility is less than or equal to a credibility threshold. The supplementary scene information includes scene information of the target scene acquired after the current time period, and the acquisition strategy for the supplementary scene information differs from the acquisition strategy for the first scene information. The second determination module 950 is further configured to: update the current credibility of the abnormal state based on the supplementary scene information. The processing module 960 is further configured to: process the abnormal state when the updated current credibility is greater than the credibility threshold; and jump to the step of acquiring supplementary scene information related to the abnormal state when the updated current credibility is less than or equal to the credibility threshold.

[0165] In one possible implementation, the second determining module 950 is further configured to: update the state change chain of the abnormal state based on the supplementary scene information; obtain the second clarity of the image information in the supplementary scene information; determine the second rationality of the formation process of the abnormal state based on the updated state change chain; determine the second accuracy of the supplementary scene information based on the second clarity; and determine the current credibility based on the second rationality and the second accuracy.

[0166] In one possible implementation, the processing module 960 is configured to: analyze the state change chain of the abnormal state to obtain abnormal impact information of the abnormal state, wherein the abnormal impact information is the impact range and abnormal result of the abnormal state obtained by predicting the abnormal state; determine the abnormal level of the abnormal state based on the abnormal impact information; generate a handling plan for the abnormal state based on the abnormal impact information and the abnormal level of the abnormal state, wherein the handling plan includes at least one of a prompting plan, an observation plan, and an intervention plan; and handle the abnormal state according to the handling plan.

[0167] In one possible implementation, the processing module 960 is further configured to: in response to the processing scheme including the observation scheme, determine the acquisition strategy corresponding to the observation scheme; acquire third-scene information related to the abnormal state according to the acquisition strategy; update the state change chain of the abnormal state based on the third-scene information; update the abnormal impact information of the abnormal state according to the updated state change chain; update the abnormal level of the abnormal state according to the updated abnormal impact information; update the processing scheme based on the updated abnormal impact information and the updated abnormal level; and process the abnormal state based on the updated processing scheme.

[0168] In one possible implementation, the acquisition module 910 is further configured to: acquire historical scene information within the target scene during a second time period, wherein the second time period is the time period preceding the first time period, and the duration of the second time period is longer than the duration of the first time period; the first determination module 940 is further configured to: determine the scene patterns of the target scene based on the historical scene information; wherein the scene patterns include at least one of the spatial layout patterns, object distribution patterns, object activity patterns, and object association patterns of the target scene; the scene abnormal state monitoring device further includes: a construction module, configured to construct baseline scene information based on the scene patterns.

[0169] Exemplary electronic devices Figure 10 This is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure, including at least one processor 2100 and a memory 2200.

[0170] The processor 2100 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 2000 to perform desired functions.

[0171] The memory 2200 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 2100 may execute one or more computer program instructions to implement the abnormal state monitoring method and / or other desired functions in the various embodiments of this disclosure described above.

[0172] In one example, the electronic device 2000 may also include an input device 2300 and an output device 2400, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0173] The input device 2300 may also include, for example, a keyboard, a mouse, etc.

[0174] The output device 2400 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0175] Of course, for the sake of simplicity, Figure 10 Only some of the components of the electronic device 2000 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 2000 may include any other suitable components depending on the specific application.

[0176] Exemplary computer program products and computer-readable storage media In addition to the methods and devices described above, embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the abnormal state monitoring methods for various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0177] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0178] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the abnormal state monitoring methods for various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0179] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0180] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0181] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0182] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0183] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0184] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for monitoring abnormal states in a scenario, the method comprising: Collect first scene information of the target scene, the first scene information including scene information of the target scene collected within the current time period; The first scene information is compared with the baseline scene information of the target scene to obtain the information difference value. The baseline scene information is the scene information that is stored in advance when the target scene is in a normal state. If the information difference value is greater than or equal to the difference threshold, it is determined that there is an abnormal state in the target scene; Collect second scene information related to the abnormal state, the second scene information including scene information of the target scene collected in a first time period before the current time period; The current credibility of the abnormal state is determined based on the first scenario information and the second scenario information; If the current credibility level is greater than the credibility threshold, the abnormal state is processed.

2. The abnormal state monitoring method according to claim 1, wherein, Determining the current credibility of the abnormal state based on the first scenario information and the second scenario information includes: Based on the first scenario information and the second scenario information, a state change chain of the abnormal state is determined, and the state change chain is used to characterize the formation process of the abnormal state; Obtain the first sharpness of the image information in the first scene information and the second scene information; Based on the state change chain, determine the first reasonableness of the formation process of the abnormal state; Based on the first clarity, determine the first accuracy of the first scene information and the second scene information; Based on the first reasonableness and the first accuracy, the current credibility is determined.

3. The abnormal state monitoring method according to claim 2, wherein, After determining the current credibility of the abnormal state based on the first scenario information and the second scenario information, the method further includes: If the current credibility is less than or equal to the credibility threshold, supplementary scene information related to the abnormal state is collected. The supplementary scene information includes scene information of the target scene collected after the current time period. The collection strategy of the supplementary scene information is different from the collection strategy of the first scene information. Based on the supplementary scenario information, update the current credibility of the abnormal state; If the updated current credibility is greater than the credibility threshold, the abnormal state is processed. If the updated current credibility is less than or equal to the credibility threshold, proceed to the step of collecting supplementary scenario information related to the abnormal state.

4. The abnormal state monitoring method according to claim 3, wherein, The step of updating the current credibility of the abnormal state based on the supplementary scenario information includes: Based on the supplementary scenario information, update the state change chain of the abnormal state; Obtain the second sharpness of the image information in the supplementary scene information; Based on the updated state change chain, a second reasonableness for the formation process of the abnormal state is determined; Based on the second clarity, determine the second accuracy of the supplementary scene information; The current credibility is determined based on the second reasonableness and the second accuracy.

5. The abnormal state monitoring method according to any one of claims 2 to 4, wherein, The process of handling the abnormal state includes: The state change chain of the abnormal state is analyzed to obtain the abnormal impact information of the abnormal state, which is the impact range and abnormal result of the abnormal state obtained by predicting the abnormal state. Based on the abnormal impact information, determine the abnormality level of the abnormal state; Based on the abnormal impact information and the abnormal level of the abnormal state, a handling plan for the abnormal state is generated, and the handling plan includes at least one of the following: a prompting plan, an observation plan, and an intervention plan. The abnormal state is handled according to the aforementioned processing scheme.

6. The abnormal state monitoring method according to claim 5, wherein, The step of processing the abnormal state according to the processing scheme includes: In response to the processing scheme including the observation scheme, a collection strategy corresponding to the observation scheme is determined; According to the acquisition strategy, third scenario information related to the abnormal state is acquired; Based on the third scenario information, update the state change chain of the abnormal state; Based on the updated state change chain, update the abnormal impact information of the abnormal state; Based on the updated abnormal impact information, update the abnormal level of the abnormal state; Based on the updated anomaly impact information and the updated anomaly level, the processing plan is updated; The abnormal state is handled based on the updated processing scheme.

7. The abnormal state monitoring method according to claim 1, wherein, Before collecting the first scene information of the target scene, the method further includes: During the second time period, historical scene information within the target scene is collected. The second time period is the time period prior to the first time period, and the duration of the second time period is longer than the duration of the first time period. Based on the historical scene information, determine the scene patterns of the target scene; The scene patterns include at least one of the following: spatial layout patterns, object distribution patterns, object activity patterns, and object association patterns of the target scene. Based on the described scene patterns, the baseline scene information is constructed.

8. An abnormal state monitoring device for a scenario, the device comprising: The acquisition module is used to acquire first scene information and second scene information of the target scene. The first scene information includes scene information of the target scene acquired within the current time period, and the second scene information includes scene information of the target scene acquired within the first time period before the current time period. A memory module is used to store baseline scene information, which is pre-stored scene information corresponding to the target scene when it is in a normal state; The comparison module is used to compare the first scene information with the baseline scene information to obtain the information difference value; The first determining module is used to determine that an abnormal state exists in the target scene when the information difference value is greater than or equal to the difference threshold. The second determining module is used to determine the current credibility of the abnormal state based on the first scenario information and the second scenario information; The processing module is used to process the abnormal state when the current credibility is greater than the credibility threshold.

9. A computer-readable storage medium storing a computer program for executing an abnormal state monitoring method for a scenario according to any one of claims 1 to 7.

10. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the abnormal state monitoring method according to any one of claims 1 to 7.

Images