Abnormality processing method, apparatus, device, storage medium, and program product

Abstract

The application provides an abnormality processing method, device, equipment, storage medium and program product. It relates to the field of data processing. The method comprises the following steps: acquiring a plurality of image data collected by a monitoring device in a monitoring area; performing labeling processing on the plurality of image data to obtain labeling information corresponding to the plurality of image data, wherein the labeling information is used to indicate features contained in the image data; determining current condition information according to the plurality of image data and the labeling information corresponding to the plurality of image data, wherein the current condition information is used to indicate whether an abnormal condition exists; and when the current condition information indicates that an abnormal condition exists, sending the current condition information to a target device of an inspection personnel. The efficiency of abnormality processing is improved.

Description

Technical Field

[0001] This application relates to the field of data processing, and more particularly to an anomaly handling method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the development of science and technology, key management areas can be monitored around the clock and across the entire region to ensure regional security.

[0003] In related technologies, surveillance equipment can be deployed in key management areas to record video and transmit the video data to the monitoring room. Monitoring personnel can then investigate abnormal behavior based on the video content from the surveillance equipment.

[0004] However, in the above methods, relying on manual handling of abnormal behavior leads to low efficiency in abnormal handling. Summary of the Invention

[0005] This application provides an exception handling method, apparatus, device, storage medium, and program product to solve the problem that manual exception handling leads to low efficiency.

[0006] Firstly, this application provides an exception handling method, including:

[0007] Acquire multiple image data collected by monitoring equipment in the monitored area;

[0008] Multiple image data are labeled to obtain the labeling information corresponding to the multiple image data. The labeling information is used to indicate the features contained in the image data.

[0009] Based on multiple image data and the corresponding annotation information, the current status information is determined, which is used to indicate whether there is an abnormal situation.

[0010] When the current status information indicates an anomaly, the current status information will be sent to the target equipment of the inspection personnel.

[0011] In one possible design, the current status information is determined based on multiple image data and the corresponding annotation information, including:

[0012] Determine if there are any abnormal annotations in the annotation information;

[0013] If there are abnormal annotations in the annotation information, determine the abnormal image corresponding to the abnormal annotations;

[0014] Determine the current situation information based on the abnormal image.

[0015] In one possible design, based on the anomaly image, current situation information is determined, including:

[0016] Input the abnormal images and abnormal annotation information into the first model;

[0017] Receive the first status information output by the first model;

[0018] Determine the first parameter of the first situation information, the first parameter being used to indicate the urgency of the first situation information;

[0019] Based on the first parameter and the first status information, determine the current status information.

[0020] In one possible design, the current status information is determined based on the first parameter and the first status information, including:

[0021] When the first parameter indicates that the first status information is a non-emergency situation, the current status information is determined to be without anomalies;

[0022] When the first parameter indicates that the first situation information is an emergency situation, the abnormal image, abnormal annotation information, first situation information and first parameter are input into the second model to obtain the second situation information and handling instructions output by the second model. The processing capacity of the second model is greater than that of the first model. The handling instructions are used to instruct the inspection personnel on how to handle the situation.

[0023] The second status information and the handling instructions are determined as the current status information.

[0024] In one possible design, sending current status information to the target equipment of the inspection personnel includes:

[0025] Determine the location of the anomaly corresponding to the current status information;

[0026] Obtain the location information of the first device that is in an idle state. The first device is the terminal device of the inspection personnel.

[0027] Based on the abnormal location and the positioning information of the first device, the target device is identified in the first device, and the current status information is sent to the target device of the inspection personnel.

[0028] In one possible design, the target device is determined within the first device based on the abnormal location and the positioning information of the first device, including:

[0029] Determine the distance between the abnormal location and the location information of the first device;

[0030] Based on distance, the second device is determined from the first device, and the second device is closest to the abnormal location;

[0031] The second device is identified as the target device.

[0032] Secondly, this application provides an anomaly handling apparatus, comprising: an acquisition module, an annotation processing module, a determination module, and a sending module, wherein,

[0033] The acquisition module is used to acquire multiple image data collected by the monitoring equipment in the monitored area;

[0034] The annotation processing module is used to annotate multiple image data to obtain annotation information corresponding to the multiple image data. The annotation information is used to indicate the features contained in the image data.

[0035] The determination module is used to determine the current status information based on multiple image data and the corresponding annotation information of the multiple image data. The current status information is used to indicate whether there is an abnormal situation.

[0036] The sending module is used to send the current status information to the target equipment of the inspection personnel when the current status information indicates an anomaly.

[0037] In one possible design, the module is specifically used for,

[0038] Determine if there are any abnormal annotations in the annotation information;

[0039] If there are abnormal annotations in the annotation information, determine the abnormal image corresponding to the abnormal annotations;

[0040] Determine the current situation information based on the abnormal image.

[0041] In one possible design, the module is specifically used for,

[0042] Input the abnormal images and abnormal annotation information into the first model;

[0043] Receive the first status information output by the first model;

[0044] Determine the first parameter of the first situation information, the first parameter being used to indicate the urgency of the first situation information;

[0045] Based on the first parameter and the first status information, determine the current status information.

[0046] In one possible design, the module is specifically used for,

[0047] When the first parameter indicates that the first status information is a non-emergency situation, the current status information is determined to be without anomalies;

[0048] When the first parameter indicates that the first situation information is an emergency situation, the abnormal image, abnormal annotation information, first situation information and first parameter are input into the second model to obtain the second situation information and handling instructions output by the second model. The processing capacity of the second model is greater than that of the first model. The handling instructions are used to instruct the inspection personnel on how to handle the situation.

[0049] The second status information and the handling instructions are determined as the current status information.

[0050] In one possible design, the sending module is specifically used for,

[0051] Determine the location of the anomaly corresponding to the current status information;

[0052] Obtain the location information of the first device that is in an idle state. The first device is the terminal device of the inspection personnel.

[0053] Based on the abnormal location and the positioning information of the first device, the target device is identified in the first device, and the current status information is sent to the target device of the inspection personnel.

[0054] In one possible design, the sending module is specifically used for,

[0055] Determine the distance between the abnormal location and the location information of the first device;

[0056] Based on distance, the second device is determined from the first device, and the second device is closest to the abnormal location;

[0057] The second device is identified as the target device.

[0058] Thirdly, this application provides an electronic device, comprising: at least one processor and a memory; the memory storing computer-executable instructions; and at least one processor executing the computer-executable instructions stored in the memory, such that the at least one processor performs the exception handling methods described in the first aspect above and various possible designs of the first aspect.

[0059] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the exception handling methods described in the first aspect and various possible designs of the first aspect.

[0060] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the exception handling method described in the first aspect above and various possible designs of the first aspect.

[0061] The anomaly handling method, apparatus, device, storage medium, and program product provided in this application can acquire multiple image data collected by monitoring equipment in the monitored area when monitoring of key areas is required; perform annotation processing on the multiple image data to obtain annotation information corresponding to the multiple image data, which is used to indicate the features contained in the image data; determine the current status information based on the multiple image data and the corresponding annotation information, which is used to indicate whether there is an anomaly; and send the current status information to the target equipment of the inspection personnel when the current status information indicates that an anomaly exists. Through the above method, the image data collected by the monitoring equipment can be pre-analyzed based on the annotation information to obtain annotation information. The current status information can then be determined based on the annotation information and the image data. When an anomaly is indicated, the current status information is sent to the inspection personnel for processing. This eliminates the need for manual real-time video inspection, improving the efficiency of anomaly handling. Furthermore, the pre-analysis of annotation information avoids subsequent full analysis of the image data, further improving the efficiency of anomaly handling and saving computing resources. Attached Figure Description

[0062] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0063] Figure 1 This is a schematic diagram of the system architecture provided for an embodiment of this application;

[0064] Figure 2 A flowchart illustrating an exception handling method provided in an embodiment of this application;

[0065] Figure 3 This is a schematic diagram illustrating the process of determining current status information provided in an embodiment of this application;

[0066] Figure 4 A schematic diagram illustrating the process of determining the target device provided in this application embodiment;

[0067] Figure 5 This is a schematic diagram of the structure of an exception handling device provided in an embodiment of this application;

[0068] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0069] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0070] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0071] The collection, storage, use, processing, transmission, provision, and disclosure of user data and other information involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0072] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0073] To facilitate understanding, the following will be combined with... Figure 1 The system architecture applicable to the embodiments of this application will be described.

[0074] Figure 1 This is a schematic diagram of the system architecture provided for an embodiment of this application. Please refer to [link / reference]. Figure 1 This includes monitoring equipment, electronic equipment, and target devices. Monitoring equipment can be devices deployed in the monitored area, such as cameras. Electronic equipment can be devices with on-device computing capabilities, such as terminal devices or servers. Target devices can be handheld terminal devices used by inspection personnel, such as mobile phones or recorders. The monitoring equipment can send the collected video data to the electronic equipment via a switch. The electronic equipment can extract multiple discrete image data from the video data according to a preset frame rate. The electronic equipment can analyze and process the image data to determine if any abnormalities exist. After confirming the existence of an abnormality, the electronic equipment can send the anomaly information to the target device used by the inspection personnel so that the personnel can handle the anomaly.

[0075] In related technologies, surveillance equipment can be deployed in key management areas to record video and transmit the video data to the monitoring room. Monitoring personnel can then investigate abnormal behavior based on the video content. However, in the above methods, relying on manual handling of abnormal behavior leads to low efficiency in anomaly detection.

[0076] To address the aforementioned technical issues, in this embodiment, when monitoring a key area is required, multiple image data collected by the monitoring equipment in the monitored area can be acquired. These multiple image data are then labeled to obtain corresponding labeling information, which indicates the features contained within the image data. Based on the multiple image data and their corresponding labeling information, current status information is determined, indicating whether any abnormalities exist. When the current status information indicates an anomaly, it is sent to the target device of the inspection personnel. This method allows for pre-analysis of the image data collected by the monitoring equipment based on the labeling information, thereby obtaining the labeling information. The current status information is then determined based on the labeling information and the image data. When an anomaly is indicated, the current status information is sent to the inspection personnel for processing. This eliminates the need for manual real-time video inspection, improving the efficiency of anomaly handling. Furthermore, the pre-analysis of the labeling information avoids subsequent full analysis of the image data, further improving anomaly handling efficiency and saving computational resources.

[0077] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0078] Figure 2 This is a flowchart illustrating an exception handling method provided in an embodiment of this application. Please refer to... Figure 2 As shown, the method may include the following steps:

[0079] S201. Obtain multiple image data collected by the monitoring equipment in the monitored area.

[0080] The execution subject of this application embodiment can be an electronic device or an exception handling device installed in the electronic device. The exception handling device can be implemented by software or by a combination of software and hardware.

[0081] A monitored area refers to an area that requires security monitoring, such as a ticket hall, turnstiles, or plaza. Monitored areas typically have a high level of security protection and may have high population density and complex situations, thus requiring intermediate management.

[0082] Monitoring equipment can be hardware devices deployed in the monitored area to collect video or image data. For example, monitoring equipment can be network cameras, panoramic cameras, etc.

[0083] Multiple image data can be image data directly collected by monitoring equipment, for example, the monitoring equipment can capture one picture per second; multiple image data can also be image data extracted by electronic equipment based on video data collected by monitoring equipment according to a preset frame rate, for example, after receiving video data collected by monitoring equipment, electronic equipment can extract image data at a frequency of 10 frames per minute.

[0084] S202. Perform annotation processing on multiple image data to obtain annotation information corresponding to multiple image data.

[0085] Labeling processing refers to the identification, classification, and labeling of key features within image data. In essence, labeling processing can convert the content of an image into text content.

[0086] In one possible implementation, annotation processing can be performed using an image annotation model. This model can be trained using a variety of pre-set samples and their corresponding annotation information.

[0087] In one possible implementation, the image annotation model can be trained to adapt to security scenario domains. For example, when annotating image data, static feature data such as gender and vehicle color are not considered; instead, dynamic feature data, such as movement of people, tipped-over objects, and illegally parked vehicles, are prioritized. This reduces the amount of data processed and improves the ability to focus on security anomalies.

[0088] In one possible implementation, the annotation process can also use object detection algorithms, such as the You Only Look Once (YOLO) algorithm, to process the image data and automatically mark the effective information in the image. Then, an image comparison algorithm is used to compare the changes in the target position between the previous N frames and the next M frames of the image data to determine the annotation information.

[0089] Annotation information can be the output of annotation processing, that is, a structured description of the content of image data. Annotation information can be used to indicate the features contained in the image data. For example, annotation information can be "there is a person who has broken into the image", "there is a lost object in the image", "there is a person who has fallen in the image", etc.

[0090] Understandably, annotation processing can be a process of pre-processing image data. Through annotation processing, the total amount of image data to be processed in subsequent processing can be reduced, avoiding the waste of resources and time caused by full processing.

[0091] S203. Determine the current status information based on multiple image data and the corresponding annotation information of the multiple image data.

[0092] Current status information refers to the current status of the monitored area. This information can include any abnormal situations, specific details of the abnormalities, and the methods used to handle them. Current status information can be used to indicate whether any abnormal situations exist.

[0093] In one possible implementation, the current status information can be determined as follows: determine whether there is abnormal annotation information in the annotation information; if there is abnormal annotation information in the annotation information, determine the abnormal image corresponding to the abnormal annotation information; and determine the current status information based on the abnormal image.

[0094] Anomaly annotations can be information in the annotation data that indicates features that do not conform to normal standards. For example, anomaly annotations may include water accumulation in the environment, material spillage, vehicles entering restricted areas, or people lying on the ground. Understandably, based on the annotation information in the image data, a preset lookup table can be used to determine whether the annotation information is anomaly. The preset lookup table indicates the comparison relationship between annotation information and whether it is anomaly. For example, when the annotation information is "There are people standing in the image," the preset lookup table can determine that the annotation information is not anomaly. When the annotation information is "There is a fight among people in the image," the preset lookup table can determine that the annotation information is anomaly.

[0095] An abnormal image can refer to image data that contains abnormal annotation information, that is, the image contains abnormal features, such as an image annotated with equipment leaking oil, or an image showing an open flame, etc.

[0096] S204. When the current status information indicates an anomaly, the current status information is sent to the target equipment of the inspection personnel.

[0097] The target device can be a terminal device for inspection personnel in the monitored area to receive current status information. The target device can have communication and information display functions. For example, the target device can be a mobile phone, smartwatch, handheld inspection terminal, etc.

[0098] In one possible implementation, the target device can be determined as follows: determine the abnormal location corresponding to the current status information; obtain the location information of a first device that is in an idle state, the first device being the inspection personnel's terminal device; determine the target device in the first device based on the abnormal location and the location information of the first device, and send the current status information to the inspection personnel's target device.

[0099] An abnormal location can refer to the specific coordinates or location information of the abnormal situation. The abnormal location can be determined based on the preset location information of the monitoring equipment. For example, the abnormal location can be the south gate parking lot, or it can be the latitude and longitude coordinates (113.9°E, 22.5°N).

[0100] The first device can also refer to the terminal device of the inspection personnel. An idle state can mean that the inspection personnel corresponding to the first device are not currently handling other abnormal tasks, or that the inspection personnel corresponding to the first device are in a standby state. This can be determined by obtaining the task list or device location of the first device. That is, when the task list of the first device is empty or there are no tasks being executed, the first device can be considered to be in an idle state. Alternatively, if the location information of the first device does not change within a preset time period, for example, if the location information of the first device does not change within five minutes, then the first device can also be considered to be in an idle state.

[0101] Location information can refer to the real-time geographical location data of the first device. Location information can be obtained through the Global Positioning System (GPS), BeiDou positioning, or base station positioning. Location information can be used to determine the distance between inspection personnel and abnormal locations.

[0102] In this embodiment, when monitoring a key area is required, multiple image data collected by monitoring equipment in the monitored area can be acquired. The monitored area can refer to the area requiring security monitoring. The image data can be image data directly collected by the monitoring equipment, or image data extracted by electronic devices based on video data collected by the monitoring equipment at a preset frame rate. Multiple image data are labeled to obtain corresponding labeling information. Labeling can refer to identifying, classifying, and marking key features within the image data. The labeling information can be a structured description of the image data content. Based on the multiple image data and their corresponding labeling information, current status information is determined. This current status information refers to the current status of the monitored area. When the current status information indicates an anomaly, it is sent to the target device of the inspection personnel. The target device can be the terminal device of the inspection personnel in the monitored area receiving the current status information. Thus, this method eliminates the need for manual real-time video inspection, improving the efficiency of anomaly handling and reducing labor costs. Furthermore, by pre-analyzing the labeling information, subsequent full analysis of the image data is avoided, further improving the efficiency of anomaly handling and saving computational resources.

[0103] Based on any of the above embodiments, the following, in conjunction with Figure 3 The process of determining the current situation information is explained in detail.

[0104] Figure 3 This is a schematic diagram illustrating the process of determining current status information provided in an embodiment of this application. Please refer to [link / reference needed]. Figure 3 The method may include:

[0105] S301. Determine whether there are any abnormal annotation information in the annotation information.

[0106] Anomaly annotation information refers to information in the indicator annotation information that indicates features that do not conform to normal standards.

[0107] Understandably, if abnormal annotations are found, further analysis of the image data is required. Conversely, if no abnormal annotations are found, it indicates no anomalies and continuous monitoring and analysis are possible. This way, annotation information can pre-filter out some normal image data, reducing the amount of subsequent analysis and processing.

[0108] S302. When abnormal annotation information exists in the annotation information, determine the abnormal image corresponding to the abnormal annotation information.

[0109] An abnormal image can refer to image data annotated with abnormal information. It can be understood that any image data corresponds to at least one annotation information. That is, the abnormal image corresponding to the abnormal annotation information can be determined based on the correspondence between image data and annotation information.

[0110] For example, suppose the anomaly label information is "there is a cluster of people in the image", then the image data corresponding to the anomaly label information may contain images of a large number of people.

[0111] Understandably, abnormal images can be used for more detailed and specific analysis later, and can also assist in manual verification of the authenticity of the anomalies.

[0112] S303. Determine the current status information based on the abnormal image.

[0113] In one possible implementation, the current situation information can be determined as follows: inputting the abnormal image and abnormal annotation information into the first model; receiving the first situation information output by the first model; determining the first parameter of the first situation information, the first parameter being used to indicate the urgency of the first situation information; and determining the current situation information based on the first parameter and the first situation information.

[0114] The first model can be a pre-trained lightweight model, such as a model based on the MobileNet architecture. The first model can be mounted on an electronic device or on an external model training server.

[0115] The first model can more accurately determine the situation information of the image based on the pre-annotation processing. For example, when the abnormal annotation information is "people gathering", the first model can further analyze the input abnormal annotation information and abnormal image, and the output first situation information can be "an accident occurred in the square, causing abnormal gathering of people". The corresponding first parameter can indicate the urgency level of the first situation information as urgent.

[0116] Understandably, since the first model is a lightweight model, it runs quickly and consumes few resources, making it a good way to initially determine the situation information.

[0117] In one possible implementation, when the first parameter indicates that the first situation information is a non-emergency situation, the current situation information is determined to be that there is no anomaly; when the first parameter indicates that the first situation information is an emergency situation, the abnormal image, abnormal annotation information, the first situation information and the first parameter are input into the second model to obtain the second situation information and handling instructions output by the second model; the second situation information and handling instructions are determined as the current situation information.

[0118] The second model can be a large language model, meaning that the processing power of the second model is greater than that of the first model. The second model can be a model based on the ResNet architecture of residual networks.

[0119] Understandably, when the first parameter indicates that the first status information is a non-emergency situation, it can be assumed that the impact of the anomaly is small, and no on-site handling by the inspection personnel is required, thus confirming that the current status information is not abnormal.

[0120] The handling instructions can be used to direct the inspection personnel on how to handle the situation. That is, the specific operational guidelines output by the second model for the anomaly. For example, the handling instructions can clearly state what the inspection personnel should do, how to do it, and precautions. For example, the handling instructions could be "Evacuate the crowd and calm their emotions."

[0121] It is understandable that by simultaneously inputting the abnormal image, abnormal annotation information, first condition information, and first parameter into the second model, since the abnormal annotation information, first condition information, and first parameter are already the results of annotation processing and analysis by the first model, the analysis and processing content of the second model can be reduced when inputting them into the second model, thereby further improving the analysis accuracy and efficiency of the second model.

[0122] For example, assuming the anomaly label information is "fire lane is blocked in the image", and the anomaly image is an image of multiple cardboard boxes placed in the fire lane, the anomaly label information and the anomaly image can be input into the first model. The first model can output the first status information as "multiple cardboard boxes are blocked in the fire lane" and the first parameter as "non-emergency status", and determine that the current status information is no anomaly.

[0123] For example, assuming the anomaly label information is "open flame in the image" and the anomaly image is an image of an open flame in the image, the anomaly label information and the anomaly image can be input into the first model. The first model can output the first situation information as "large-scale open flame in the square" and the first parameter as "emergency situation". Then, the anomaly image, anomaly label information, first situation information and first parameter can be input into the second model to obtain the second situation information and handling instructions output by the second model. The second situation information can be "the flower bed in the square is on fire and needs to be handled urgently by the patrol personnel", and the handling instructions can be "please have the patrol personnel call the fire department and organize the evacuation of the crowd".

[0124] In one possible implementation, audio data collected by the monitoring device corresponding to the abnormal image can also be obtained. The audio data, along with the abnormal image, abnormal annotation information, first condition information, and first parameters, can be input into the second model. This multimodal data fusion method can improve the recognition capability of the second model and enhance the monitoring effect of abnormal behavior in complex scenarios. For example, in low-light scenarios at night, when the video image clarity is low, the audio data can be combined for analysis to improve the accuracy of recognition.

[0125] In one possible implementation, environmental information can also be obtained from the image data. This environmental information can be used to indicate the current pedestrian flow. For example, when the environmental information indicates an increase in pedestrian flow, the computing power allocation of the first model can be increased so that the first model can quickly filter out anomalies with less impact. When the environmental information indicates a decrease in pedestrian flow, the computing power allocation of the second model can be increased so that the second model can improve the recognition accuracy of anomalies with greater impact and the accuracy of handling instructions.

[0126] exist Figure 3 In the illustrated embodiment, when it is necessary to determine the current status information, it can be determined whether there is abnormal annotation information in the annotation information. Abnormal annotation information can refer to information in the annotation information whose indicator features do not conform to normal standards. When abnormal annotation information exists in the annotation information, the abnormal image corresponding to the abnormal annotation information is determined. The abnormal image can refer to image data with abnormal information. Based on the abnormal image, the current status information can be determined through a dual-model implementation using a first model and a second model. In this way, hierarchical processing can be achieved. First, abnormal images with abnormal annotation information are filtered out based on the annotation processing. Then, a preliminary analysis is performed based on the lightweight first model to determine the urgency of the abnormality. In an emergency, a deeper analysis is performed using the second model to output more detailed second status information and handling instructions, enabling inspection personnel to handle the abnormality. This avoids the problems of insufficient analysis by a single lightweight model and excessive model resource consumption. At the same time, generating handling instructions directly through the second model can avoid mishandling problems caused by relying on human experience.

[0127] Based on any of the above embodiments, the following, in conjunction with Figure 4 The process of determining the target equipment is described in detail.

[0128] Figure 4 This is a schematic diagram illustrating the process of determining the target device provided in an embodiment of this application. Please refer to... Figure 4 The method may include:

[0129] S401. Determine the location of the anomaly corresponding to the current status information.

[0130] The abnormal location can be the specific location where the abnormality occurred, corresponding to the current status information. The abnormal location can be represented by spatial coordinates.

[0131] In one possible implementation, the image data corresponding to the current situation information can be determined, the target monitoring device that captured the image data can be determined based on the identification information of the image data, and the location information and coverage area of ​​the target monitoring device can be determined. The location information and coverage area can then be used to determine the abnormal location corresponding to the current situation information.

[0132] In one possible implementation, after determining the location information and coverage of the target monitoring device, the relative position of the anomaly in the image data can be determined by image recognition, and then the anomaly location can be accurately located based on the location information and coverage of the target monitoring device.

[0133] S402, Obtain the location information of the first device that is in an idle state.

[0134] The first device can be the terminal device of each inspection personnel. That is, any idle terminal device of the inspection personnel within the monitoring range can be considered as the first device.

[0135] "Idle status" can refer to the situation where the inspection personnel corresponding to the first device are not currently undertaking other inspection tasks or have completed their tasks and are in a standby state.

[0136] The location information of the first device can be obtained through the positioning device inside the first device, or through the Global Positioning System (GPS), BeiDou positioning, or base station positioning.

[0137] S403. Based on the abnormal location and the positioning information of the first device, identify the target device in the first device and send the current status information to the target device of the inspection personnel.

[0138] The target device can be the first device ultimately selected to receive current status information.

[0139] In one possible implementation, the target device can be determined as follows: determine the distance between the abnormal location and the location information of the first device; based on the distance, determine the second device among the first devices, the second device being the closest to the abnormal location; and determine the second device as the target device.

[0140] The distance between the abnormal location and the location information of the first device can refer to either the straight-line distance or the actual travel distance between them. Optionally, in small-scale scenarios with minimal obstruction, such as squares or ticket halls, the straight-line distance can be used as the distance between the abnormal location and the location information of the first device. In large-scale scenarios with road constraints, such as parks, highways, or residential areas, the actual travel distance can be used as the distance between the abnormal location and the location information of the first device.

[0141] In one possible implementation, the route information from the target device's location to the abnormal location can be determined using a path planning algorithm based on the abnormal location and the target device's location information. This route information can be used to indicate the shortest route for the inspection personnel carrying the target device to reach the abnormal location.

[0142] exist Figure 4 In the illustrated embodiment, when it is necessary to determine the target device, the abnormal location corresponding to the current status information can be determined. The abnormal location can be the specific location where the abnormality occurred. The location information of a first device in an idle state can be obtained. The first device can be the terminal device of each inspection personnel. Based on the abnormal location and the location information of the first device, the target device is determined among the first devices, and the current status information is sent to the target device of the inspection personnel. In this way, by dispatching tasks to the target device with distance priority, the idle inspection personnel closest to the abnormal location can take over the task first, thereby improving the efficiency of abnormality handling. At the same time, selecting different ranging methods based on different scenarios can further improve the accuracy of task dispatch.

[0143] Figure 5 This is a schematic diagram of an exception handling device provided in an embodiment of this application. Please refer to... Figure 5 The anomaly handling device 10 includes: an acquisition module 11, a labeling processing module 12, a determination module 13, and a sending module 14, wherein,

[0144] The acquisition module 11 is used to acquire multiple image data collected by the monitoring equipment in the monitoring area;

[0145] The annotation processing module 12 is used to perform annotation processing on multiple image data to obtain annotation information corresponding to the multiple image data. The annotation information is used to indicate the features contained in the image data.

[0146] The determination module 13 is used to determine the current status information based on multiple image data and the annotation information corresponding to the multiple image data. The current status information is used to indicate whether there is an abnormal situation.

[0147] The sending module 14 is used to send the current status information to the target equipment of the inspection personnel when the current status information indicates that there is an abnormality.

[0148] The exception handling device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0149] In one possible design, module 13 is specifically used for,

[0150] Determine if there are any abnormal annotations in the annotation information;

[0151] If there are abnormal annotations in the annotation information, determine the abnormal image corresponding to the abnormal annotations;

[0152] Determine the current situation information based on the abnormal image.

[0153] In one possible design, module 13 is specifically used for,

[0154] Input the abnormal images and abnormal annotation information into the first model;

[0155] Receive the first status information output by the first model;

[0156] Determine the first parameter of the first situation information, the first parameter being used to indicate the urgency of the first situation information;

[0157] Based on the first parameter and the first status information, determine the current status information.

[0158] In one possible design, module 13 is specifically used for,

[0159] When the first parameter indicates that the first status information is a non-emergency situation, the current status information is determined to be without anomalies;

[0160] When the first parameter indicates that the first situation information is an emergency situation, the abnormal image, abnormal annotation information, first situation information and first parameter are input into the second model to obtain the second situation information and handling instructions output by the second model. The processing capacity of the second model is greater than that of the first model. The handling instructions are used to instruct the inspection personnel on how to handle the situation.

[0161] The second status information and the handling instructions are determined as the current status information.

[0162] In one possible design, the sending module 14 is specifically used for,

[0163] Determine the location of the anomaly corresponding to the current status information;

[0164] Obtain the location information of the first device that is in an idle state. The first device is the terminal device of the inspection personnel.

[0165] Based on the abnormal location and the positioning information of the first device, the target device is identified in the first device, and the current status information is sent to the target device of the inspection personnel.

[0166] In one possible design, the sending module 14 is specifically used for,

[0167] Determine the distance between the abnormal location and the location information of the first device;

[0168] Based on distance, the second device is determined from the first device, and the second device is closest to the abnormal location;

[0169] The second device is identified as the target device.

[0170] The exception handling device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0171] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 20 may include: a transceiver 21, a processor 22, and a memory 23.

[0172] Processor 22 executes computer execution instructions stored in memory, causing processor 22 to perform the scheme in the above embodiments. Processor 22 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0173] The memory 23 is connected to the processor 22 via the system bus and completes communication between them. The memory 23 is used to store computer program instructions.

[0174] Transceiver 21 can be used to obtain the task to be run and the configuration information of the task to be run.

[0175] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0176] The electronic device provided in this application embodiment can be the terminal device described in the above embodiments.

[0177] This application also provides a chip for executing instructions, which is used to execute the technical solution of the exception handling method in the above embodiments.

[0178] This application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the exception handling method described in the above embodiments.

[0179] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the technical solution of the exception handling method in the above embodiments.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0181] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0182] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0183] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0184] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0185] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0186] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0187] The aforementioned storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0188] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.

[0189] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An exception handling method, characterized in that, include: Acquire multiple image data collected by monitoring equipment in the monitored area; The plurality of image data are labeled to obtain the labeling information corresponding to the plurality of image data, and the labeling information is used to indicate the features contained in the image data; Based on the multiple image data and the corresponding annotation information, the current status information is determined, which is used to indicate whether there is an abnormal situation; When the current status information indicates an anomaly, the current status information is sent to the target equipment of the inspection personnel.

2. The method according to claim 1, characterized in that, Based on the multiple image data and the corresponding annotation information, the current status information is determined, including: Determine whether there is any abnormal annotation information in the annotation information; If abnormal annotation information exists in the annotation information, determine the abnormal image corresponding to the abnormal annotation information; Based on the abnormal image, determine the current situation information.

3. The method according to claim 2, characterized in that, Based on the abnormal image, the current situation information is determined, including: Input the abnormal image and the abnormal annotation information into the first model; Receive the first status information output by the first model; Determine a first parameter for the first situation information, wherein the first parameter is used to indicate the urgency level of the first situation information; Based on the first parameter and the first status information, determine the current status information.

4. The method according to claim 3, characterized in that, Based on the first parameter and the first status information, the current status information is determined, including: When the first parameter indicates that the first status information is a non-emergency situation, the current status information is determined to be without anomalies; When the first parameter indicates that the first situation information is an emergency situation, the abnormal image, the abnormal annotation information, the first situation information and the first parameter are input into the second model to obtain the second situation information and handling instructions output by the second model. The processing capacity of the second model is greater than that of the first model. The handling instructions are used to instruct the inspection personnel on how to handle the situation. The second status information and the handling instruction are determined as the current status information.

5. The method according to any one of claims 1-4, characterized in that, Sending the current status information to the target equipment of the inspection personnel includes: Determine the location of the anomaly corresponding to the current status information; Obtain the location information of the first device that is in an idle state, wherein the first device is the terminal device of the inspection personnel; Based on the abnormal location and the positioning information of the first device, the target device is determined in the first device, and the current status information is sent to the target device of the inspection personnel.

6. The method according to claim 5, characterized in that, Based on the abnormal location and the positioning information of the first device, the target device is determined within the first device, including: Determine the distance between the abnormal location and the positioning information of the first device; Based on the distance, a second device is determined among the first devices, the second device being the closest to the abnormal location; The second device is identified as the target device.

7. An anomaly handling device, characterized in that, include: The module consists of an acquisition module, an annotation processing module, a determination module, and a sending module. The acquisition module is used to acquire multiple image data collected by the monitoring equipment in the monitoring area; The annotation processing module is used to perform annotation processing on the plurality of image data to obtain annotation information corresponding to the plurality of image data, wherein the annotation information is used to indicate the features contained in the image data; The determining module is used to determine current status information based on the plurality of image data and the annotation information corresponding to the plurality of image data, wherein the current status information is used to indicate whether there is an abnormal situation; The sending module is used to send the current status information to the target equipment of the inspection personnel when the current status information indicates an anomaly.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images