State identification method, device, electronic device and storage medium

Abstract

The present application provides a state recognition method, including: obtaining a manhole cover area image of a manhole cover; determining the state of the manhole cover area based on the manhole cover area image; and sending a prompt message to a background server when the state of the manhole cover area is a dangerous state. The present application determines the state of the manhole cover area by using the manhole cover area image, and when the state of the manhole cover area is a dangerous state, sends a prompt message to the background server, which sends the prompt message to the target device. The maintenance personnel corresponding to the target device will then perform manhole cover maintenance, thereby ensuring the safety of the manhole cover, improving the efficiency of manhole cover inspection, and being low-cost, and conducive to improving user experience. The present application also provides a state recognition device, an electronic device, and a computer-readable storage medium, all of which have the above-mentioned beneficial effects.

Description

Technical Field

[0001] The present application relates to the field of identification technology, and in particular to a state identification method, device, electronic device and storage medium. Background Art

[0002] Urban sewer systems are a fundamental and crucial public welfare project for a city. As the entrance to the system, sewers are typically covered with manhole covers to prevent accidents. Checking for missing or damaged manhole covers requires significant manpower and resources, and any oversight can easily create safety hazards.

[0003] Heavy rains have recently hit several provinces in southern China, leaving many cities facing severe flooding. Rainwater mixing with sewage systems, lifting manhole covers, and pedestrians, unable to see clearly, often fall into them. However, cameras located at high altitudes can easily detect the swirling flow of water through lifted manhole covers, making it easier to distinguish between normal standing and flowing water. Therefore, traffic system cameras can detect and report abnormal sewer openings.

[0004] At the same time, map-based urban monitoring has been implemented in the Safe City project. In addition to the main transportation hubs in the city, cameras on main roads and main roads in residential areas can basically detect the location of sewer manhole covers. These sewer manhole covers in the main road area have a wide field of view and are places where pedestrians must pass. Pedestrians are not easily alert and often become the main cause of accidents.

[0005] Related technologies implement manhole cover monitoring systems based on cloud platforms, using Internet of Things technology. These systems connect to the cloud through base stations and electronic tags inside manhole covers, enabling functions such as 24 / 7 monitoring, cover opening alarms, and intelligent water level detection. Alternatively, hydraulic manhole covers can be used. However, this method requires a large amount of hardware. For just one manhole cover, sensors for monitoring the water level and manhole cover are needed, as well as terminals for lock control and detection. There are also requirements for the material of the manhole cover, resulting in excessive overhead. At the same time, the waterproofness and airtightness of the sensors, as well as the power consumption of the terminal equipment, need to be considered. In the event of a power outage, manual replacement and maintenance are required, which is labor-intensive and costly to replace.

[0006] Therefore, how to provide a solution to the above technical problems is a problem that those skilled in the art need to solve at present. Summary of the Invention

[0007] The purpose of this application is to provide a state recognition method, device, electronic device, and storage medium that can improve manhole cover detection efficiency and reduce costs. The specific solution is as follows:

[0008] This application provides a state identification method, including:

[0009] Obtaining a manhole cover area image of the manhole cover;

[0010] determining a state of the manhole cover area according to the manhole cover area image;

[0011] When the state of the manhole cover area is dangerous, a prompt message is sent to the background server.

[0012] Preferably, determining the state of the manhole cover area according to the manhole cover area image includes:

[0013] Judging whether the position of the manhole cover is normal according to the manhole cover area image;

[0014] If the position of the manhole cover is normal, determining the damage condition of the manhole cover according to the manhole cover area image, and determining the state of the manhole cover area according to the damage condition;

[0015] If the position of the manhole cover is abnormal, determining whether there is a water flow feature in the manhole cover area image;

[0016] If the water flow feature does not exist, determining the state of the manhole cover area according to the current manhole cover position in the manhole cover area image;

[0017] If the water flow characteristics exist, the state of the manhole cover area is determined according to the water flow characteristics.

[0018] Preferably, determining the state of the manhole cover area according to the water flow characteristics includes:

[0019] When the water flow characteristic is outward flow and the manhole cover is not completely installed with the sewer opening, it is determined that the state of the manhole cover area is a non-dangerous state;

[0020] When the water flow characteristic is a spring, and there is no manhole cover at the sewer outlet, it is determined that the state of the manhole cover area is a dangerous state;

[0021] When the water flow characteristic is a vortex water flow, it is determined that the state of the manhole cover area is a dangerous state;

[0022] When the water flow characteristic is that the water flows into the sewer along the edge of the sewer, the state of the manhole cover area is determined according to the pre-stored manhole cover image.

[0023] Preferably, determining the damage condition of the manhole cover according to the manhole cover area image includes:

[0024] Matching the manhole cover area image with a pre-stored manhole cover image to obtain a matching result;

[0025] The damage condition of the manhole cover is determined according to the matching result.

[0026] Preferably, the position of the manhole cover is normal including: the manhole cover is completely installed with the sewer opening;

[0027] The abnormal position of the manhole cover includes: the manhole cover is not at the sewer outlet, and the manhole cover is not completely installed with the sewer outlet.

[0028] Preferably, the determining whether there is a water flow feature in the manhole cover area image includes:

[0029] A pre-established water flow recognition model is used according to the manhole cover area image to determine whether water flow features exist.

[0030] Preferably, when the state of the manhole cover area is in a dangerous state, after sending a prompt message to the backend server, the method further includes:

[0031] Get the current manhole cover area image;

[0032] Determining whether there is an object close to the manhole cover according to the current manhole cover area image;

[0033] When the object exists, an alarm instruction is sent to issue an alarm.

[0034] The present application provides a state recognition device, comprising:

[0035] An image acquisition module, used to acquire an image of a manhole cover area of ​​the manhole cover;

[0036] A state determination module, configured to determine the state of the manhole cover area based on the manhole cover area image;

[0037] The sending module is used to send a prompt message to the background server when the state of the manhole cover area is dangerous.

[0038] The present application provides an electronic device, comprising:

[0039] memory for storing computer programs;

[0040] A processor is configured to implement the steps of the above-mentioned state identification method when executing the computer program.

[0041] The present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the state identification method described above are implemented.

[0042] The present application provides a state recognition method, including: obtaining a manhole cover area image of a manhole cover; determining the state of the manhole cover area based on the manhole cover area image; and sending a prompt message to a background server when the state of the manhole cover area is a dangerous state.

[0043] It can be seen that this application determines the status of the manhole cover area by utilizing the image of the manhole cover area. When the status of the manhole cover area is dangerous, a prompt message is sent to the background server. The background server sends the prompt message to the target device, and then the maintenance personnel corresponding to the target device will perform manhole cover maintenance to ensure the safety of the manhole cover, improve the efficiency of manhole cover inspection, low cost, and help improve user experience.

[0044] The present application also provides a state recognition device, an electronic device and a computer-readable storage medium, all of which have the above-mentioned beneficial effects and will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0046] Figure 1 A flow chart of a state identification method provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of a water flow characteristic provided in an embodiment of the present application is water outflow;

[0048] Figure 3 A schematic diagram of a water flow characterized by a spring-like flow is provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of a water flow characterized by a vortex water flow provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of a water flow characteristic provided for an embodiment of the present application is a flow along the edge of a sewer into a sewer;

[0051] Figure 6 A schematic diagram of an image of a manhole cover area provided in an embodiment of the present application;

[0052] Figure 7 A schematic diagram of the positions of a manhole cover and a speaker provided in an embodiment of the present application;

[0053] Figure 8 A schematic diagram of an alarm provided in an embodiment of the present application;

[0054] Figure 9 This is a structural diagram of a state recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0056] Related technologies implement manhole cover monitoring systems based on cloud platforms, using IoT technology. These systems connect to the cloud through base stations and electronic tags inside manhole covers, enabling functions such as 24 / 7 monitoring, cover opening alarms, and intelligent water level detection. Alternatively, hydraulic manhole covers can be used. However, this approach requires significant hardware. Just for a single manhole cover, sensors for monitoring the water level and manhole cover are required, as well as terminals for lock control and detection. There are also requirements for the material of the manhole cover, resulting in excessive overhead. Furthermore, the waterproofness and airtightness of the sensors, as well as the power consumption of the terminal equipment, must be considered. In the event of a power outage, manual replacement and maintenance are required, which is labor-intensive and costly.

[0057] Based on the above technical problems, this embodiment provides a state recognition method to improve the efficiency of manhole cover inspection with low cost. Figure 1 , Figure 1 A flow chart of a state identification method provided in an embodiment of the present application specifically includes:

[0058] S101, obtaining a manhole cover area image of the manhole cover;

[0059] In this embodiment, a camera is provided near the manhole cover for collecting the manhole cover area image of the manhole cover. The manhole cover area image includes the sewer image at the sewer position corresponding to the manhole cover. This embodiment does not limit the position of the sewer image in the manhole cover area image. The position of the sewer image collected each time in the entire manhole cover area image may be the same or different. Preferably, the camera collects the manhole cover area image at a preset angle and position. The manhole cover may be a metal manhole cover, a high-strength fiber cement concrete manhole cover, or a resin manhole cover. The manhole cover area involved in this embodiment mainly refers to the area with a radius of 2 meters with the manhole cover as the center captured by the camera,

[0060] S102, determining the state of the manhole cover area according to the manhole cover area image;

[0061] The purpose of this step is to determine the state of the manhole cover area, where the state of the manhole cover area includes dangerous state and non-dangerous state.

[0062] Determining the status of the manhole cover area based on the manhole cover area image can be based on the position of the manhole cover in the manhole cover area image, the damage condition of the manhole cover; it can also be based on the position of the manhole cover in the manhole cover area image, the damage condition of the manhole cover and the water flow characteristics.

[0063] In one achievable embodiment, determining the manhole cover area status based on the manhole cover area image includes determining the manhole cover area status based on the position of the manhole cover and the damage status of the manhole cover in the manhole cover area image. Specifically, the manhole cover area image determines whether the position of the manhole cover is normal; if the position of the manhole cover is normal, the damage status of the manhole cover is determined based on the manhole cover area image; and the manhole cover area status is determined based on the damage status. A normal manhole cover position includes: the manhole cover is fully installed at the sewer opening; and a manhole cover position includes: the manhole cover is not at the sewer opening, or the manhole cover is not fully installed at the sewer opening. Determining the damage condition of the manhole cover based on the manhole cover area image includes: calculating the similarity between the manhole cover area image and a pre-stored standard manhole cover area image to obtain a similarity value; determining the damage condition based on the similarity value, specifically, when the similarity value is greater than a preset similarity threshold, determining the damage condition as conventional damage, wherein the conventional damage indicates that the manhole cover is damaged but does not affect normal use or is not damaged; when the similarity value is not greater than the preset similarity threshold, determining the damage condition as severe damage, wherein the severe damage indicates that the degree of damage to the manhole cover has reached a level that affects normal use. Furthermore, when the damage condition is severe damage, the manhole cover area is in a dangerous state, and when the damage condition is conventional damage, the manhole cover area is in a non-dangerous state.

[0064] In another achievable embodiment, determining the state of the manhole cover area based on the manhole cover area image includes: determining the state of the manhole cover area based on the position of the manhole cover, the damage condition of the manhole cover, and the water flow characteristics in the manhole cover area image. Specifically, based on the manhole cover area image, determining whether the position of the manhole cover is normal; if the position of the manhole cover is normal, determining the damage condition of the manhole cover based on the manhole cover area image, and determining the state of the manhole cover area based on the damage condition; if the position of the manhole cover is abnormal, determining whether there is a water flow characteristic in the manhole cover area image; if there is no water flow characteristic, determining the state of the manhole cover area based on the current position of the manhole cover in the manhole cover area image; if there is a water flow characteristic, determining the state of the manhole cover area based on the water flow characteristic.

[0065] S103: When the manhole cover area is in a dangerous state, a prompt message is sent to the backend server.

[0066] When the manhole cover area is in a dangerous state, a prompt message is sent to the backend server, which then sends the prompt message to the target device. The corresponding maintenance personnel of the target device then perform manhole cover maintenance. The prompt message includes, but is not limited to, the manhole cover identification or camera identification, the manhole cover area image (image or video), the manhole cover area status, and the prompt content. The manhole cover identification uniquely corresponds to the manhole cover location, and the manhole cover identification can be the manhole cover number, QR code, barcode, etc. The camera identification is associated with the camera location.

[0067] When the current state of the manhole cover area is determined to be dangerous, that is, the sewer area corresponding to the manhole cover has been divided into a dangerous area, the background server sends the determined image of the manhole cover area to the server (target device), and the server pushes it to the corresponding person in charge. The person in charge makes a secondary manual judgment based on the image of the manhole cover area. After determining that there is indeed a problem, he goes to repair or prevent it according to the given camera point, and automatically restores the area status (from the dangerous area to the normal area) after completion.

[0068] Based on the above technical solution, this embodiment determines the status of the manhole cover area by utilizing the manhole cover area image. When the status of the manhole cover area is dangerous, a prompt message is sent to the background server. The background server sends the prompt message to the target device, and then the maintenance personnel corresponding to the target device will perform manhole cover maintenance to ensure the safety of the manhole cover, improve the efficiency of manhole cover inspection, reduce costs, and help improve user experience.

[0069] Furthermore, in order to improve the accuracy of state recognition, S102 includes:

[0070] S1021. Determine whether the position of the manhole cover is normal based on the manhole cover area image;

[0071] In this embodiment, features in the manhole cover area image may be identified to obtain feature information, where the feature information includes but is not limited to one or more of manhole cover position features, manhole cover shape features, and water flow features. In this step, a judgment is made based on the manhole cover position features in the feature information of the area image to determine whether the position of the manhole cover is normal. The position of the manhole cover may be completely installed at the sewer outlet or not completely installed at the sewer outlet. The incomplete installation of the manhole cover at the sewer outlet includes: the manhole cover is in contact with the sewer outlet but not completely installed, and the manhole cover is not in contact with the sewer outlet. When there is no contact, the manhole cover image may be in the manhole cover area image or not in the manhole cover area image. In this step, the position of the manhole cover is normal only when the manhole cover is completely installed at the sewer outlet; otherwise, the position of the manhole cover is abnormal.

[0072] S1022: If the position of the manhole cover is normal, determine the damage of the manhole cover based on the image of the manhole cover area, and determine the state of the manhole cover area based on the damage;

[0073] When the position of the manhole cover is normal, the state of the manhole cover is checked, that is, the damage of the manhole cover is determined.

[0074] It is understandable that when the manhole cover is damaged, it needs to be replaced. At this time, the status of the manhole cover area is in a dangerous state. Further, a prompt message can be sent to the background server, wherein the prompt message includes prompt content, and the prompt content can be "The manhole cover is damaged and needs to be replaced". Of course, it can also be other content. This embodiment will no longer limit it, as long as it can achieve the purpose of this embodiment.

[0075] S1023. If the position of the manhole cover is abnormal, determine whether there is a water flow feature in the manhole cover area image;

[0076] When the position of the manhole cover is abnormal, it is determined whether there is a water flow feature in the manhole cover area image, that is, whether there is water flow in the manhole cover area. Among them, the water flow feature can be a spring of water gushing out of the sewer outlet and a water outflow, or it can be a vortex of water flowing into the sewer outlet and water flowing into the sewer along the edge of the sewer; of course, there may be other forms, which are not limited in this embodiment. Specifically, it can be determined whether the features extracted from the manhole cover area image have water flow features, or the manhole cover area image can be input into a pre-established water flow recognition model to determine whether there is a water flow feature. Of course, other methods are also possible, which are not elaborated in this embodiment, as long as they can achieve the purpose of this embodiment.

[0077] Furthermore, the determining whether there is a water flow feature in the manhole cover area image includes:

[0078] A pre-established water flow recognition model is used according to the manhole cover area image to determine whether water flow features exist.

[0079] Specifically, a neural network model is trained using training images with and without different water flow characteristics to generate a water flow recognition model. Inputting an image of a manhole cover area into the water flow recognition model can output whether a water flow characteristic exists. Furthermore, when a water flow characteristic exists, the model can output whether the specific flow characteristic is a vortex, a water flow, or a spring. This embodiment does not limit the algorithm corresponding to the neural network model, and users can customize it.

[0080] S1024: If there is no water flow feature, determining the state of the manhole cover area according to the current manhole cover position in the manhole cover area image;

[0081] Among them, the current manhole cover position refers to the location of the manhole cover. It can be understood that when there is a manhole cover image in the manhole cover area image, the manhole cover area status can be determined based on the current manhole cover position. When there is no manhole cover image in the manhole cover area image, the current manhole cover position is none, and it is determined that the manhole cover is lost. At this time, the manhole cover area status is a dangerous state. Specifically, determining the manhole cover area status based on the current manhole cover position in the manhole cover area image can include: comparing and analyzing the current manhole cover position in an image recognition manner based on the initial manhole cover standard image stored in the database to determine whether the manhole cover is displaced or lost. Further, a prompt message can be sent to the background server, wherein the prompt message includes prompt content, and the prompt content can be "The manhole cover position at the sewer is abnormal, please go and solve it."

[0082] S1025. If there are water flow characteristics, determine the state of the manhole cover area based on the water flow characteristics.

[0083] If there are water flow characteristics, the water flow algorithm is used to determine the status of the manhole cover area based on the water flow characteristics.

[0084] It can be seen that this embodiment accurately determines the status of the manhole cover area through the position, damage condition and water flow characteristics of the manhole cover, thereby improving the accuracy of status recognition.

[0085] Furthermore, the state of the manhole cover area is determined based on the water flow characteristics, including:

[0086] When the water flow characteristic is outflow and the manhole cover is not fully installed with the sewer outlet, the manhole cover area is determined to be in a non-dangerous state;

[0087] When the water flow characteristic is a spring, and there is no manhole cover at the sewer outlet, the manhole cover area is determined to be in a dangerous state;

[0088] When the water flow characteristic is a vortex water flow, the state of the manhole cover area is determined to be a dangerous state;

[0089] When the water flow characteristic is that the water flows into the sewer along the edge of the sewer, the state of the manhole cover area is determined according to the pre-stored manhole cover image.

[0090] Among them, the water flow forms water accumulation in the sewer area (manhole cover area), which is divided into four stages.

[0091] In the early stage, when rainwater enters the sewer, the pressure inside the sewer is too high, and when it is intense, the manhole cover will be lifted up. At this time, the iron manhole cover weighing hundreds of pounds will often be lifted up, and a "bouncing" shape will appear at the sewer outlet. The water flow characteristics in the early stage are that there is a large amount of liquid flowing out from under the manhole cover (water flow outflow), and the manhole cover is not completely installed at the sewer outlet. For details, please refer to Figure 2 , Figure 2A schematic diagram of a water flow characteristic provided in an embodiment of the present application shows outward water flow, which determines the manhole cover area to be in a non-hazardous state. Furthermore, an initial prompt message can be sent to the backend server, where the prompt message includes the prompt content and the manhole cover location. The prompt content may be "There is a risk of manhole cover displacement or even water accumulation. It is recommended to make early warning deployments in the surrounding area."

[0092] In the middle stage, the rainwater has lifted the manhole cover and caused it to shift, and the water in the sewer gushes out. At this time, the water flow is like a spring. The corresponding water flow characteristics are water flow springs. For details, please refer to Figure 3 , Figure 3 A schematic diagram of a water flow characterized by a spring-like flow is provided in an embodiment of the present application.

[0093] Specifically, the process of determining the water flow characteristics as a water spring is as follows: when the water flow meets the first condition, the second condition and the third condition, the water flow characteristics are determined to be a water spring. Among them, the first condition is that a large amount of liquid gushes out from the sewer as the center, the second condition is that the top of the water column is higher than the ground and the diameter is basically the same as the sewer outlet, and the third condition is that the water column is perpendicular to the horizontal plane. Different from the situations in which there are fire hydrant ruptures in the area (the bottom end is higher than the ground), water guns (the direction of the water column is parallel to the ground, not vertically upward; the bottom end of the water column is not on the ground), etc., the upward spraying water column caused by the rupture of the sewer pipe (the diameter of the water column is thinner than in this case). It can be understood that the amount of water flowing out of the sewer outlet per unit time during a water spring is greater than the amount of water when the water flows outward. When the water flow characteristics are gushing water and the manhole cover is displaced, the manhole cover area is determined to be in a dangerous state. Among them, the manhole cover displacement means that the manhole cover is not in the sewer outlet area. At this time, the prompt content in the prompt information can be "The manhole cover is no longer in the normal position, the water in the sewer is overflowing, please solve it in time."

[0094] In the later stage, the accumulated water has already formed and the water is deep, and the water surface is relatively calm. The accumulated water flows back through the sewer, and the water flow is vortex-shaped. The water flow characteristics at this time are vortex water flow. For details, please refer to Figure 4 , Figure 4 A schematic diagram of a water flow characterized by a vortex water flow is provided in an embodiment of the present application.

[0095] The specific process for determining a vortex flow is as follows: A vortex flow is determined when the flow meets the fourth, fifth, and sixth conditions. The fourth condition is that the water waves cover the entire sewer area; the fifth condition is that the water surface is concave in the middle, with no manhole cover visible; and the sixth condition is that multiple circular waves oriented in a single direction form a spiraling surface centered on the sewer. For further details on vortex flow, please refer to Table 1, which provides a table of vortex flow conditions.

[0096] Table 1 is the vortex water flow situation table

[0097]

[0098] Therefore, based on the judgment of the fourth, fifth and sixth conditions, the late stage and danger are accurately identified. At this time, the prompt content in the prompt information can be "water accumulation in the area, vortex water flow is detected, and it is suspected that the manhole cover is no longer in the normal position. Please confirm and solve it in time."

[0099] Finally, when most of the accumulated water recedes and the remaining accumulated water recedes from the sewer, the water flows into the sewer from the edge of the sewer due to the shallowness of the remaining accumulated water, and no vortex can be formed. At this time, the water flow characteristic is that the water flows into the sewer along the edge of the sewer. For details, please refer to Figure 5 , Figure 5 A water flow feature provided in an embodiment of the present application is a schematic diagram of water flowing into a sewer along its edge. The state of the manhole cover area is then determined based on a pre-stored manhole cover image. Specifically, the presence of a manhole cover at the sewer opening and its position (completely covering the sewer opening) are compared with images of normal manhole covers stored in a database. Furthermore, a final prompt message can be sent to the backend server. The final prompt message may include the following: "The manhole cover position at the sewer is abnormal. Please go and resolve it."

[0100] Furthermore, this embodiment provides a specific method for determining the damage condition of the manhole cover based on the manhole cover area image, including:

[0101] Matching the manhole cover area image with the pre-stored manhole cover image to obtain a matching result;

[0102] Determine the damage condition of the manhole cover based on the matching results.

[0103] The pre-stored manhole cover image is a standard image of a fully assembled manhole cover. Specifically, the matching process can calculate the similarity between the manhole cover area image and the pre-stored manhole cover image, and determine a matching result based on the similarity. If the similarity exceeds a preset threshold, the match is considered successful; otherwise, the match fails. The degree of damage is determined based on the matching result: a successful match indicates the manhole cover is intact, while a failed match indicates the manhole cover is damaged. Other possible implementations are not limited in this embodiment.

[0104] It can be seen that this embodiment determines the damage condition by matching the manhole cover area image with the pre-stored manhole cover image, with high accuracy.

[0105] Furthermore, obtaining the manhole cover area image of the manhole cover includes: sending an acquisition instruction to the camera so that the camera acquires the manhole cover area image according to the angle and pixel position in the acquisition instruction; and obtaining the manhole cover area image sent by the camera.

[0106] The camera captures the manhole cover and divides the sewer area (manhole cover area) according to the manhole cover. The camera at a high position captures the manhole cover within the visual range, that is, the one-meter radius around the identified manhole cover is divided into the manhole cover area, and its corresponding angle and the position of the four corner pixels are remembered. The subsequent judgment and capture of the manhole cover are only performed on this manhole cover area. For details, please refer to Figure 6 , Figure 6 A schematic diagram of a manhole cover area image provided in an embodiment of the present application.

[0107] Furthermore, in order to ensure that the manhole cover area image including the manhole cover image can be accurately captured, it also includes: regularly updating the angle and pixel position.

[0108] Among them, because the mechanical position of the camera may be offset due to uncontrollable factors, the point information of the sewer area needs to be updated regularly. Specifically, whether to update the image of the manhole cover captured within a day or update it at a longer interval needs to be selected and set according to the situation. Most urban flooding can be eliminated within the same day, so regularly updating the point information can narrow the point gap caused by camera offset. At the same time, the horizontal distance of one meter can also make the image capture have a higher fault tolerance rate. It can be seen that this embodiment ensures that the image of the manhole cover area can be accurately obtained by regularly updating the angle and pixel position.

[0109] From the above, we can see that although the probability of a manhole cover being lifted by rainwater is small, once it happens, the probability of injury to pedestrians is 100%. For related technologies, it is not necessary to spend a lot of effort to replace manhole covers and regularly maintain and replace smart manhole covers to solve this 1% small probability problem. This embodiment solves the danger problem without adding hardware facilities. At the same time, the safe city has complete facilities, mature algorithms, and mature cloud servers in terms of video surveillance terminals. It has inherent advantages in solving the problem of dangerous manhole covers without additional costs. This embodiment can alarm for damaged manhole covers in normal times and prompt the terminal to replace them. In rainy weather, it can provide early warning of whether the manhole cover is lifted by accumulated water, resulting in an unprotected sewer. At the same time, due to the existing dangerous area determination technology, it can capture people entering the dangerous area and immediately sound a buzzer alarm, which is much more efficient than the current staff's investigation and setting up warning signs.

[0110] Furthermore, in order to improve the user experience and provide a prompt for a dangerous state, when the manhole cover area is in a dangerous state, after sending a prompt message to the backend server, it also includes:

[0111] Get the current manhole cover area image;

[0112] Determine whether there is an object close to the manhole cover based on the current manhole cover area image;

[0113] When an object is present, an alarm command is sent to issue an alarm.

[0114] In this embodiment, in a dangerous situation, it takes time for the staff to arrive at the scene. If someone accidentally enters the dangerous area during this time difference, their safety cannot be protected. Therefore, this embodiment sends a warning instruction to warn, thereby ensuring the safety of the user. In a feasible implementation, a warning device is set up on site, which includes but is not limited to a waterproof speaker / buzzer (depending on the budget). The specific warning device can be placed under the sewer manhole cover, such as Figure 7 As shown, Figure 7 A schematic diagram of the position of a manhole cover and a speaker provided in an embodiment of the present application. When the camera recognizes that an object is approaching a dangerous area, it immediately links the remote control of the warning device at the sewer to warn people to take a detour and not get close. The remote control method depends on the sensor to select a specific solution, such as wireless remote control (the upcoming 5G has great advantages in this regard), Bluetooth remote control, etc. Figure 8 As shown, Figure 8 A schematic diagram of an alarm provided in an embodiment of the present application.

[0115] In another feasible implementation, the manhole cover in this embodiment includes an alarm module. When the manhole cover area is in a dangerous state and an object approaches the manhole cover, a warning instruction is sent to the alarm module of the manhole cover so that the alarm module can issue an alarm.

[0116] It can be seen that this embodiment ensures that when an object approaches a dangerous area, a warning instruction is sent to the warning device through communication between the warning device and the camera, so that the warning device issues a warning, thereby ensuring the safety of the user.

[0117] The following is an introduction to a state recognition device provided in an embodiment of the present application. The state recognition device described below and the state recognition method described above can be referred to in correspondence with each other. Figure 9 , Figure 9 A schematic diagram of the structure of a state recognition device provided in an embodiment of the present application includes:

[0118] An image acquisition module 201 is used to acquire an image of a manhole cover area of ​​the manhole cover;

[0119] A state determination module 202 is used to determine the state of the manhole cover area based on the manhole cover area image;

[0120] The sending module 203 is used to send a prompt message to the background server when the state of the manhole cover area is dangerous.

[0121] Preferably, the state determination module 202 includes:

[0122] A position determination unit, configured to determine whether the position of the manhole cover is normal based on the manhole cover area image;

[0123] a damage condition determination unit, configured to determine the damage condition of the manhole cover according to the manhole cover area image if the position of the manhole cover is normal, and determine the state of the manhole cover area according to the damage condition;

[0124] A water flow feature existence judgment unit is used to judge whether there is a water flow feature in the manhole cover area image if the position of the manhole cover is abnormal;

[0125] a first state determining unit, configured to determine the state of the manhole cover area according to the current manhole cover position in the manhole cover area image if no water flow feature exists;

[0126] The second state determination unit is used to determine the state of the manhole cover area according to the water flow characteristics if there are water flow characteristics.

[0127] Preferably, the second state determining unit includes:

[0128] The first state determination subunit is configured to determine that the state of the manhole cover area is a non-dangerous state when the water flow characteristic is outflow and the manhole cover is not fully installed with the sewer opening;

[0129] The second state determination subunit is used to determine that the state of the manhole cover area is a dangerous state when the water flow characteristic is a spring and there is no manhole cover at the sewer outlet;

[0130] The third state determination subunit is used to determine that the state of the manhole cover area is a dangerous state when the water flow characteristic is a vortex water flow;

[0131] The fourth state determination subunit is used to determine the state of the manhole cover area according to the pre-stored manhole cover image when the water flow characteristic is that the water flows into the sewer along the edge of the sewer.

[0132] Preferably, the damage condition determination unit includes:

[0133] The matching subunit is used to match the manhole cover area image with the pre-stored manhole cover image to obtain a matching result;

[0134] The damage condition determination subunit is used to determine the damage condition of the manhole cover based on the matching result.

[0135] Preferably, the normal position of the manhole cover includes: the manhole cover is completely installed with the sewer opening;

[0136] Abnormal manhole cover positions include: no manhole cover at the sewer outlet, and the manhole cover is not fully installed with the sewer outlet.

[0137] Preferably, the water flow feature existence judgment unit includes:

[0138] The water flow feature existence judgment subunit is used to judge whether the water flow feature exists based on the manhole cover area image using a pre-established water flow recognition model.

[0139] Preferably, it also includes:

[0140] The current manhole cover area image acquisition module is used to obtain the current manhole cover area image;

[0141] The proximity judgment module is used to judge whether there is an object close to the manhole cover based on the current manhole cover area image;

[0142] The warning instruction sending module is used to send a warning instruction to warn when an object exists.

[0143] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, and they will not be repeated here.

[0144] An electronic device provided in an embodiment of the present application is introduced below. The electronic device described below and the method described above can refer to each other.

[0145] The present application provides an electronic device, comprising:

[0146] memory for storing computer programs;

[0147] A processor is used to implement the steps of the above state identification method when executing a computer program.

[0148] Since the embodiments of the electronic device part correspond to the embodiments of the method part, the embodiments of the electronic device part refer to the description of the embodiments of the method part, which will not be repeated here.

[0149] A computer-readable storage medium provided in an embodiment of the present application is introduced below. The computer-readable storage medium described below and the method described above can be referenced to each other.

[0150] The present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above state identification method are implemented.

[0151] Since the embodiments of the computer-readable storage medium part correspond to the embodiments of the method part, the embodiments of the computer-readable storage medium part refer to the description of the embodiments of the method part and are not repeated here.

[0152] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0153] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0154] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0155] The above is a detailed introduction to a state recognition method, device, electronic device, and medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A state recognition method, characterized in that: include: Obtaining a manhole cover area image of the manhole cover; determining a state of the manhole cover area according to the manhole cover area image; When the state of the manhole cover area is dangerous, a prompt message is sent to the backend server; The determining the state of the manhole cover area according to the manhole cover area image includes: Judging whether the position of the manhole cover is normal according to the manhole cover area image; If the position of the manhole cover is normal, determining the damage condition of the manhole cover according to the manhole cover area image, and determining the state of the manhole cover area according to the damage condition; If the position of the manhole cover is abnormal, determining whether there is a water flow feature in the manhole cover area image; If the water flow feature does not exist, determining the state of the manhole cover area according to the current manhole cover position in the manhole cover area image; If the water flow characteristics exist, determining the state of the manhole cover area according to the water flow characteristics; Among them, the water flow characteristics include water flowing out of the sewer outlet like a spring, water flowing out of the sewer outlet like an outflow, vortex water flow flowing into the sewer outlet, and water flow flowing into the sewer along the edge of the sewer.

2. The state recognition method according to claim 1, characterized in that: The determining the state of the manhole cover area according to the water flow characteristics includes: When the water flow characteristic is outward flow and the manhole cover is not completely installed with the sewer opening, it is determined that the state of the manhole cover area is a non-dangerous state; When the water flow characteristic is a spring, and there is no manhole cover at the sewer outlet, it is determined that the state of the manhole cover area is a dangerous state; When the water flow characteristic is a vortex water flow, it is determined that the state of the manhole cover area is a dangerous state; When the water flow characteristic is that the water flows into the sewer along the edge of the sewer, the state of the manhole cover area is determined according to the pre-stored manhole cover image.

3. The state recognition method according to claim 1, characterized in that: The determining the damage condition of the manhole cover according to the manhole cover area image includes: Matching the manhole cover area image with a pre-stored manhole cover image to obtain a matching result; The damage condition of the manhole cover is determined according to the matching result.

4. The state recognition method according to claim 1, characterized in that: The normal position of the manhole cover includes: the manhole cover is completely installed with the sewer opening; The abnormal position of the manhole cover includes: the manhole cover is not at the sewer outlet, and the manhole cover is not completely installed with the sewer outlet.

5. The state recognition method according to claim 1, characterized in that: The determining whether there is a water flow feature in the manhole cover area image includes: A pre-established water flow recognition model is used according to the manhole cover area image to determine whether water flow features exist.

6. The state recognition method according to any one of claims 1 to 5, characterized in that: When the state of the manhole cover area is in a dangerous state, after sending a prompt message to the backend server, the method further includes: Get the current manhole cover area image; Determining whether there is an object close to the manhole cover according to the current manhole cover area image; When the object exists, an alarm instruction is sent to issue an alarm.

7. A state recognition device, characterized in that: include: An image acquisition module, used to acquire an image of a manhole cover area of ​​the manhole cover; A state determination module, configured to determine the state of the manhole cover area based on the manhole cover area image; A sending module is used to send a prompt message to the backend server when the state of the manhole cover area is dangerous; The state determination module includes: A position determination unit, configured to determine whether the position of the manhole cover is normal based on the manhole cover area image; a damage condition determination unit, configured to determine the damage condition of the manhole cover according to the manhole cover area image if the position of the manhole cover is normal, and determine the state of the manhole cover area according to the damage condition; A water flow feature existence judgment unit is used to judge whether there is a water flow feature in the manhole cover area image if the position of the manhole cover is abnormal; a first state determining unit, configured to determine the state of the manhole cover area according to the current manhole cover position in the manhole cover area image if no water flow feature exists; a second state determining unit, configured to determine the state of the manhole cover area according to the water flow characteristics if there are water flow characteristics; Among them, the water flow characteristics include water flowing out of the sewer outlet like a spring, water flowing out of the sewer outlet like an outflow, vortex water flow flowing into the sewer outlet, and water flow flowing into the sewer along the edge of the sewer.

8. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the state identification method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the state identification method according to any one of claims 1 to 6.

Images