Identification method, device and equipment for extracorporeal circulation of operation and storage medium

Abstract

The embodiment of the invention provides an operation extracorporeal circulation identification method and device, equipment and a storage medium. The method comprises the following steps: acquiring video stream data acquired by image acquisition equipment; performing identification processing on the video stream data to obtain at least one identification result corresponding to the video stream data; acquiring operation plan data corresponding to the target equipment area in a preset time period; if it is determined that all the recognition results represent that the operating personnel exist in the target equipment area within the preset time period and the operation plan data represent that the target equipment area has no operation plan within the preset time period, first early warning information is generated; wherein the first early warning information represents that the operation extracorporeal circulation exists in the target equipment area at the corresponding moment. According to the method, the operation extracorporeal circulation condition can be automatically identified, so that the identification efficiency of the operation extracorporeal circulation is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular, to a method, device, equipment, and storage medium for identifying extracorporeal circulation of operations. Background Art

[0002] With the gradual increase in the intensity of power grid transformation, the pressure of power grid security risk control has also gradually increased. Among them, most of the power grid security risks come from extracorporeal circulation of operations. Therefore, it is necessary to identify extracorporeal circulation of operations to reduce power grid security risks.

[0003] In the prior art, the extracorporeal circulation situation of each equipment area is identified by manually screening and checking item by item or going to the site for manual inspection.

[0004] However, in the above method, the manual inspection method is time-consuming and laborious, which reduces the identification efficiency of extracorporeal circulation of operations. Summary of the Invention

[0005] The embodiments of this application provide a method, device, equipment, and storage medium for identifying extracorporeal circulation of operations, which can automatically identify the extracorporeal circulation situation of operations and improve the identification efficiency of extracorporeal circulation of operations.

[0006] In a first aspect, the embodiments of this application provide a method for identifying extracorporeal circulation of operations, including:

[0007] Obtain the video stream data collected by an image acquisition device; and perform identification processing on the video stream data to obtain at least one identification result corresponding to the video stream data; wherein, the identification result includes whether there are operating personnel in the target equipment area of the target substation within a preset time period;

[0008] Obtain the operation plan data corresponding to the target equipment area within a preset time period; wherein, the operation plan data represents the operation plan situation of the target equipment area;

[0009] If it is determined that each of the identification results indicates that there are operating personnel in the target equipment area within a preset time period, and the operation plan data indicates that there is no operation plan in the target equipment area within the preset time period, then generate a first warning message; wherein, the first warning message indicates that there is extracorporeal circulation of operations in the target equipment area at the corresponding moment.

[0010] In a possible implementation manner, performing identification processing on the video stream data to obtain at least one identification result corresponding to the video stream data includes:

[0011] Perform the i-th solution processing on the video stream data to obtain the i-th image set of the video stream data; wherein, the image set includes at least one operation scene image of the target equipment area;

[0012] Perform image recognition processing on each job scenario image in the i-th image set to obtain the i-th result set; wherein, the i-th result set includes the processing result corresponding to each job scenario image; the processing result represents whether a worker appears in the target device area at the corresponding moment.

[0013] Determine the i-th recognition result according to the i-th result set.

[0014] In a possible implementation manner, performing image recognition processing on each job scenario image in the i-th image set to obtain the i-th result set includes:

[0015] Perform face recognition processing on the job scenario image to obtain the processing result corresponding to the job scenario image.

[0016] Alternatively, perform work uniform recognition processing on the job scenario image to obtain the processing result corresponding to the job scenario image.

[0017] In a possible implementation manner, determining the i-th recognition result according to the i-th result set includes:

[0018] Determine the total number of target job scenario images corresponding to the i-th image set according to the processing results in the i-th result set; wherein, the target job scenario image is a job scenario image whose processing result represents that a worker appears in the target device area at the corresponding moment.

[0019] If it is determined that the total number is greater than the preset number, determine that the i-th recognition result represents that there are workers in the target device area within the preset time period.

[0020] In a possible implementation manner, before obtaining the video stream data collected by the image acquisition device, it further includes:

[0021] Obtain the job plan data of the target device area at at least one moment.

[0022] If it is determined that the job plan data represents that there is no job plan in the target device area at the corresponding moment, or if it is determined that the job plan data represents that the job plan in the target device area is not in the execution state at the corresponding moment, then perform the step of obtaining the video stream data collected by the image acquisition device.

[0023] In a possible implementation manner, the method further includes:

[0024] If it is determined that each of the recognition results indicates the presence of operating personnel in the target equipment area within a preset time period, and it is determined that the operation plan data indicates that the operation plan for the target equipment area within the preset time period is not in an executing state, then a second warning message is generated; wherein, the second warning message indicates that unauthorized operations have occurred in the target equipment area within the preset time period.

[0025] In a possible implementation manner, the method further includes:

[0026] According to the status information of the image acquisition device, determine the health status of the video stream data; wherein, the status information indicates whether the image acquisition device is in an abnormal state; the abnormal state is an offline state or a frame rate abnormal state.

[0027] In a second aspect, an identification device for extracorporeal circulation of operations provided by an embodiment of the present application includes:

[0028] An identification module, configured to obtain video stream data collected by an image acquisition device; and perform identification processing on the video stream data to obtain at least one recognition result corresponding to the video stream data; wherein, the recognition result includes whether there are operating personnel in the target equipment area in the target substation within a preset time period;

[0029] An acquisition module, configured to acquire operation plan data corresponding to the target equipment area within a preset time period; wherein, the operation plan data indicates the operation plan situation of the target equipment area;

[0030] A warning module, configured to generate a first warning message if it is determined that each of the recognition results indicates the presence of operating personnel in the target equipment area within a preset time period and the operation plan data indicates that there is no operation plan for the target equipment area within the preset time period; wherein, the first warning message indicates the presence of extracorporeal circulation of operations in the target equipment area at the corresponding moment.

[0031] In a possible implementation manner, the identification module is specifically configured to: perform an i-th solution processing on the video stream data to obtain an i-th image set of the video stream data; wherein, the image set includes at least one operation scene image of the target equipment area; perform image recognition processing on each operation scene image in the i-th image set to obtain an i-th result set; wherein, the i-th result set includes a processing result corresponding to each operation scene image; the processing result indicates whether operating personnel appear in the target equipment area at the corresponding moment; determine the i-th recognition result according to the i-th result set.

[0032] In a possible implementation, the recognition module is specifically configured to: perform face recognition processing on the operation scenario image to obtain a processing result corresponding to the operation scenario image; or, perform work uniform recognition processing on the operation scenario image to obtain a processing result corresponding to the operation scenario image.

[0033] In a possible implementation, the recognition module is further specifically configured to: determine the total number of target operation scenario images corresponding to the i-th image set according to the processing results in the i-th result set; wherein, the target operation scenario image is an operation scenario image in which a staff member appears in the target device area at the corresponding moment; if it is determined that the total number is greater than a preset number, it is determined that the i-th recognition result indicates that there are operation personnel in the target device area within a preset time period.

[0034] In a possible implementation, before the recognition module is used to obtain the video stream data collected by the image acquisition device, the device is further configured to: obtain the operation plan data of the target device area at at least one moment; if it is determined that the operation plan data indicates that there is no operation plan in the target device area at the corresponding moment, or if it is determined that the operation plan data indicates that the operation plan in the target device area at the corresponding moment is not in an execution state, then execute the step of obtaining the video stream data collected by the image acquisition device.

[0035] In a possible implementation, the device is further configured to: if it is determined that each of the recognition results indicates that there are operation personnel in the target device area within a preset time period, and it is determined that the operation plan data indicates that the operation plan in the target device area within the preset time period is not in an execution state, then generate a second warning message; wherein, the second warning message indicates that there is an unauthorized operation situation in the target device area within the preset time period.

[0036] In a possible implementation, the device is further configured to: determine the health status of the video stream data according to the status information of the image acquisition device; wherein, the status information indicates whether the image acquisition device is in an abnormal state; the abnormal state is an offline state or a frame rate abnormal state.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0038] The memory stores computer execution instructions;

[0039] The processor executes the computer execution instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementation manners of the first aspect.

[0040] Fourthly, an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the first aspect and / or various possible implementation manners of the first aspect as described above when executed by a processor.

[0041] Fifthly, an embodiment of the present application provides a computer program product including a computer program, which implements the first aspect and / or various possible implementation manners of the first aspect as described above when executed by a processor.

[0042] The method, device, equipment and storage medium for identifying extracorporeal operation provided by the embodiments of the present application determine whether there are operators in the target device area of the target substation through continuous multiple image identifications, and compare with the operation plan situation of the target device area to push a warning message indicating the existence of extracorporeal operation in the target device area; furthermore, the extracorporeal operation situation can be automatically identified, and the identification efficiency of extracorporeal operation can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0044] Figure 1 It is a schematic flowchart of a method for identifying extracorporeal operation provided by an embodiment of the present application;

[0045] Figure 2 It is a schematic flowchart of another method for identifying extracorporeal operation provided by an embodiment of the present application;

[0046] Figure 3 It is a schematic structural diagram of a device for identifying extracorporeal operation provided by an embodiment of the present application;

[0047] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0048] Through the above-mentioned accompanying drawings, the clear embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0050] First, the terms related to the present application are explained:

[0051] Out-of-service power system operation: It refers to the operation situation in the power system where maintenance or operation is carried out without affecting normal power transmission.

[0052] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties. Moreover, the processing of relevant data, such as collection, use, processing, transmission, provision, disclosure, and application, all comply with relevant laws, regulations, and standards, adopt necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0053] And the present application involves big data analysis of user information (including but not limited to personal biometric features, identity data, consumption data, asset data, electronic terminal operation data, etc.), and uses artificial intelligence technology for automated decision-making. For technical solutions that make decisions having a significant impact on personal rights and interests based on the results of automated decision-making, corresponding operation entrances are provided for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, it enters the expert decision-making process.

[0054] It should be noted that the present application can be used in the field of image processing technology, and can also be used in any field other than image processing technology. The application field of the present application is not limited.

[0055] As the intensity of power grid transformation gradually increases, the pressure of power grid security risk control also gradually increases. Among them, most of the power grid security risks come from out-of-service power system operations. Therefore, it is necessary to identify out-of-service power system operations to reduce power grid security risks.

[0056] Combined with the above scenarios, by manually screening and checking item by item or going to the site for manual inspection to identify the out-of-service power system operation situations in each equipment area; the method of manual inspection is time-consuming and laborious, thus reducing the identification efficiency of out-of-service power system operations.

[0057] The method for identifying extracorporeal operation circulation provided by this application determines whether there are operators in the target equipment area of the target substation through continuous multiple image identifications, and compares it with the operation plan of the target equipment area, so as to push a warning message indicating the existence of extracorporeal operation circulation in the target equipment area, solving the technical problem of low identification efficiency of extracorporeal operation circulation.

[0058] The following uses specific embodiments to elaborate in detail on the technical solution of this application and how the technical solution of this application solves the above technical problems. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0059] Figure 1 It is a schematic flowchart of a method for identifying extracorporeal operation circulation provided by an embodiment of this application, as Figure 1 shown, this method includes:

[0060] 201. Obtain the video stream data collected by the image acquisition device; and perform identification processing on the video stream data to obtain at least one identification result corresponding to the video stream data; wherein, the identification result includes whether there are operators in the target equipment area of the target substation within a preset time period.

[0061] Exemplarily, the execution subject of this embodiment can be an electronic device, hereinafter referred to as the device. For each equipment area of the target substation, one or more image acquisition devices can be set. For each equipment area, the video stream data of the equipment area can be collected through the corresponding image acquisition device. The device can automatically execute tasks or, under the task start instruction of the user, obtain the video stream data from the image acquisition device. Based on video data processing technology, the obtained video stream data is parsed and identified multiple times to obtain one or more identification results. Among them, each identification result can determine whether there are operators in the target equipment area corresponding to the video stream data.

[0062] Among them, a thread pool can be used to process multiple cameras to collect video stream data. The specific code process is as follows:

[0063] def main():

[0064] camera_list = get_online_cameras() # Obtain all online cameras

[0065] with ThreadPoolExecutor(max_workers=8) as executor:

[0066] executor.map(process_camera, camera_list).

[0067] Among them, each camera can perform a full scan every 10 minutes to collect video stream data. The specific code process is: schedule.every(10).minutes.do(main).

[0068] 202. Obtain the job plan data corresponding to the target device area within a preset time period; among them, the job plan data represents the job plan situation of the target device area.

[0069] Exemplarily, the device can obtain the job plan data corresponding to the target device area within a preset time period, that is, the video stream acquisition time period, from the local database or the remote database, so as to understand the job plan situation of the target device area.

[0070] For example, combined with the ledger information such as the acquisition time of the video stream in this device area, it is possible to output information about the presence of operating personnel in a certain device area of a certain substation at a certain time point. Based on this information, inquiries can be made in the grid management platform asset domain - maintenance and repair management - query (for example, intelligent safety supervision system - job plan query) to obtain the job plan data of this device area of this substation at this time point.

[0071] Among them, the specific process based on the camera ledger docking module is as follows:

[0072] # Obtain the camera ledger from the CMDB system

[0073] def get_camera_info(camera_id):

[0074] api_url = "http: / / cmdb.grid.com / cameras"

[0075] params = {"camera_id": camera_id}

[0076] try:

[0077] response = requests.get(api_url, params=params, timeout=3)

[0078] return {

[0079] "substation": response.json()['plant'],

[0080] "area": response.json()['position'],

[0081] "geo_code": response.json()['geo_code']

[0082] }

[0083] except Exception as e:

[0084] log.error(f"Failed to obtain the ledger: {str(e)}")

[0085] return None。

[0086] 203. If it is determined that each recognition result indicates the presence of operating personnel in the target equipment area during the preset time period, and the operation plan data indicates that there is no operation plan in the target equipment area during the preset time period, then generate a first warning message; wherein, the first warning message indicates the existence of an operation external circulation in the target equipment area at the corresponding moment.

[0087] Exemplarily, the device compares each recognition result with the operation plan data. If it is determined that all recognition results indicate the presence of operating personnel in the target equipment area during the preset time period, and the operation plan data indicates that there is no operation plan in the target equipment area during the preset time period, then generate a first warning message to indicate the existence of an operation external circulation in the target equipment area at the corresponding moment.

[0088] For example, compare the two pieces of data "There are operating personnel in a certain equipment area of a certain substation at a certain time point" and "The operation plan data of this equipment area of this substation at this time point". If there are operating personnel but no corresponding operation plan, then push a warning message of "operation external circulation" in the form of voice, text, sound and light, etc.

[0089] Among them, an example of the warning message for operation external circulation is:

Intelligent Operation Monitoring System

[0090] It is worth adding that based on the multi-source verification mechanism, the crawler request interval (>30 seconds) can be set, and continuous detection can also be performed three times to avoid false alarms. The specific process is as follows:

[0091] human_count = 0

[0092] for _ in range(3):

[0093] if detect_human(img_path):

[0094] human_count += 1

[0095] time.sleep(30)

[0096] return human_count >= 2

[0097] In this embodiment, a method for identifying extracorporeal circulation of operations is provided. Through continuous multiple image recognitions, it is determined whether there are operators in the target equipment area of the target substation, and compared with the operation plan situation of the target equipment area, so as to push a warning message indicating the existence of extracorporeal circulation of operations in the target equipment area. Furthermore, the situation of extracorporeal circulation of operations can be automatically identified, improving the identification efficiency of extracorporeal circulation of operations.

[0098] Figure 2 For another flowchart of the method for identifying extracorporeal circulation of operations provided by the embodiment of the present application, as Figure 2 shown, this method includes:

[0099] 301. Obtain the video stream data collected by the image acquisition device.

[0100] Exemplarily, this step can refer to step 201, which will not be elaborated here.

[0101] In one example, before step 301, it further includes:

[0102] Step 1. Obtain the operation plan data of the target equipment area at at least one moment.

[0103] Step 2. If it is determined that the operation plan data indicates that there is no operation plan in the target equipment area at the corresponding moment, or if it is determined that the operation plan data indicates that the operation plan in the target equipment area at the corresponding moment is not in the execution state, then execute step 301.

[0104] Exemplarily, the device can first regularly obtain the operation plan data of the target equipment area at one or more moments to understand the operation plan situation of the target equipment area at the corresponding moment. If it is determined that there is no operation plan in the target equipment area at the corresponding moment, or if it is determined that the operation plan in the target equipment area at the corresponding moment is not in the execution state, then obtain the video stream data collected by the image acquisition device for processing.

[0105] For example, the data acquisition component regularly (every 30 minutes) retrieves operation plan data from the power grid management platform / smart safety supervision system; if there is no operation plan in the device area at this time point, the data acquisition component retrieves the video stream data, and further processes the video stream data in different cases through the data processing and alarm component to determine whether to push the early warning information of "operation outside the loop". It is worth adding that if there is an operation plan in a certain device area at this time point but it is not in the "executing" state, the data acquisition component retrieves the video stream data; if there is an operation plan in a certain device area at this time point and it is in the "executing" state, there is no need to retrieve the video stream data and no information needs to be pushed.

[0106] Among them, after obtaining the operation plan data, it can be cached. Based on the multi-level cache mechanism, Redis can be used to cache the operation plan data. For example:

[0107] redis_client = Redis(host='redis.grid.com', port=6379)

[0108] def check_work_plan_with_cache(substation, area):

[0109] cache_key = f"plan:{substation}:{area}"

[0110] # Give priority to reading the cache

[0111] cached_data = redis_client.get(cache_key)

[0112] if cached_data:

[0113] return json.loads(cached_data)

[0114] # Request the interface when there is no cache

[0115] fresh_data = check_work_plan_status(substation, area)

[0116] redis_client.setex(cache_key, 300, json.dumps(fresh_data)) # Cache for 5 minutes

[0117] return fresh_data.

[0118] Among them, after obtaining the job plan data, based on the job plan status parsing enhancement mechanism, the specific process is as follows:

[0119] def check_work_plan_status(substation, area):

[0120] url = f"http: / / schedule.grid.com / plans?area={area}"

[0121] try:

[0122] response = requests.get(url, timeout=10)

[0123] data = response.json()

[0124] # Find the matching plan

[0125] active_plan = next((p for p in data['plans']

[0126] if p['substation'] == substation

[0127] and p['area'] == area), None)

[0128] if not active_plan:

[0129] return "NO_PLAN" # No plan

[0130] elif active_plan['status'] == 2:

[0131] return "IN_PROGRESS" # In progress

[0132] else:

[0133] return "UNAUTHORIZED" # There is a plan not executed

[0134] except:

[0135] return "ERROR".

[0136] In one example, it further includes: determining the health status of the video stream data according to the status information of the image acquisition device; wherein, the status information indicates whether the image acquisition device is in an abnormal state; the abnormal state is an offline state or a frame rate abnormal state.

[0137] Exemplarily, after obtaining the video stream data, the device will collect the status information of the image acquisition device to determine whether the image acquisition device is in an abnormal state such as an offline state or a frame rate abnormal state, so as to judge the health status of the obtained video stream data; for example, if the image acquisition device is in an abnormal state, the obtained video stream data is not healthy, that is, the quality is poor and unavailable, then it is necessary to re-collect the video stream data or maintain the image acquisition device until the quality of the obtained video stream data is good.

[0138] For example, the code process of video stream health monitoring is as follows:

[0139] def camera_health_check():

[0140] for camera in camera_list:

[0141] rtsp_url = f"rtsp: / / {camera['ip']} / stream"

[0142] cap = cv2.VideoCapture(rtsp_url)

[0143] if not cap.isOpened():

[0144] alert_technical(f"Camera {camera['id']} is offline")

[0145] else:

[0146] fps = cap.get(cv2.CAP_PROP_FPS)

[0147] if fps < 15:

[0148] alert_technical(f"Camera {camera['id']} has abnormal frame rate")

[0149] cap.release().

[0150] 302. Perform the i-th resolution process on the video stream data to obtain the i-th image set of the video stream data; wherein, the image set includes at least one working scene image of the target device area.

[0151] Exemplarily, based on video stream processing technology, the device performs the i-th solution processing on the video stream data to obtain the i-th image set of the video stream data, including job scene images of at least one target device area.

[0152] For example, based on the video screenshot module, it can be implemented through the following code process:

[0153] import cv2

[0154] def capture_rtsp_frame(rtsp_url):

[0155] cap = cv2.VideoCapture(rtsp_url)

[0156] ret, frame = cap.read()

[0157] if ret:

[0158] timestamp = datetime.now().strftime("%Y%m%d%H%M%S")

[0159] filename = f"snapshot_{timestamp}.jpg"

[0160] cv2.imwrite(filename, frame)

[0161] cap.release()

[0162] return filename.

[0163] 303. Perform image recognition processing on each job scene image in the i-th image set to obtain the i-th result set; wherein, the i-th result set includes the processing result corresponding to each job scene image; the processing result represents whether a worker appears in the target device area at the corresponding moment.

[0164] Exemplarily, based on image recognition technology, such as an image recognition model, etc., the device performs image recognition processing on each job scene image in the i-th image set to obtain the i-th result set, including whether a worker appears in the target device area at the corresponding moment of each job scene image, that is, the processing result corresponding to each job scene image.

[0165] In one example, step 303 includes the following steps: perform face recognition processing on the job scene image to obtain the processing result corresponding to the job scene image.

[0166] Alternatively, perform work uniform recognition processing on the work scene image to obtain the processing result corresponding to the work scene image.

[0167] Exemplarily, the device performs face recognition processing on the work scene image based on face recognition technology to determine whether a worker appears in the target device area at the corresponding moment of the work scene image. For example, if it is determined that the face of a stored worker appears in the work scene image, it is determined that a worker appears in the target device area at the corresponding moment of the work scene image; otherwise, no worker appears in the target device area at the corresponding moment of the work scene image. Alternatively, based on image processing technology, perform work uniform recognition processing on the work scene image. If it is determined that a stored work uniform image appears in the work scene image, it is determined that a worker appears in the target device area at the corresponding moment of the work scene image; otherwise, no worker appears in the target device area at the corresponding moment of the work scene image.

[0168] For example, perform face recognition processing based on the human detection module. The specific code process is as follows:

[0169] import torch

[0170] model = torch.hub.load('ultralytics / yolov5', 'yolov5s')

[0171] def detect_human(img_path):

[0172] results = model(img_path)

[0173] return 'person' in results.pandas().xyxy[0]['name'].values。

[0174] 304. Determine the i-th recognition result according to the i-th result set.

[0175] Exemplarily, for each result set, that is, the processing result corresponding to each work scene image obtained after each image recognition, the corresponding recognition result can be determined through voting processing or deep learning model processing, that is, whether there is a worker in the target device area of the target substation within a preset time period after each video stream data recognition processing.

[0176] In one example, step 304 includes:

[0177] Step 1: Determine the total number of target operation scenario images corresponding to the i-th image set based on the processing results in the i-th result set; where the target operation scenario image is an operation scenario image in which a staff member appears in the target device area at the corresponding moment as represented by the processing result.

[0178] Step 2: If it is determined that the total number is greater than the preset number, it is determined that there are operating personnel in the target device area during the preset time period as represented by the i-th recognition result.

[0179] Specifically, the device determines, based on the processing results corresponding to each operation scenario image obtained in each recognition process, the operation scenario images in which personnel appear in the target device area at the corresponding moment as target operation scenario images, and counts the total number of all target operation scenario images. Among them, it can be the total number of operation scenario images in which personnel appear in the target device area at consecutive moments, or the total number of operation scenario images in which personnel appear in the target device area at any moment. The device compares this total number with the preset number. If it is determined that the total number is greater than the preset number, it means that there are staff members appearing in the target device area during the preset time period, and the number of appearances is the value corresponding to this total number, and exceeds the number corresponding to the preset number. That is, it can be determined from the recognition results obtained in this video stream data recognition process that there are operating personnel in the target device area during the preset time period. Furthermore, the recognition accuracy can be improved.

[0180] For example, based on the intelligent false alarm filtering mechanism, temporal analysis can be used to filter out temporarily passing personnel. The specific code process is as follows:

[0181] human_records = defaultdict(deque) # Record the appearance time of personnel by area

[0182] def is_real_operation(area, detect_time):

[0183] records = human_records[area]

[0184] records.append(detect_time)

[0185] # Keep the records of the most recent 1 hour

[0186] while records[0] < datetime.now() - timedelta(hours = 1):

[0187] records.popleft()

[0188] # Determine whether it persists continuously (appears 3 times within 15 minutes)

[0189] return len([t for t in records if t > datetime.now() - timedelta(minutes = 15)]) >= 3

[0190] 305. Obtain the operation plan data corresponding to the target device area within a preset time period; wherein, the operation plan data represents the operation plan situation of the target device area.

[0191] Exemplarily, this step can refer to step 202 and will not be elaborated here.

[0192] 306. If it is determined that each recognition result represents the presence of operating personnel in the target device area within a preset time period, and the operation plan data represents that there is no operation plan in the target device area within the preset time period, then generate a first warning message; wherein, the first warning message represents that there is an out-of-cycle operation in the target device area at the corresponding moment.

[0193] Exemplarily, this step can refer to step 203 and will not be elaborated here.

[0194] In one example, it further includes: If it is determined that each recognition result represents the presence of operating personnel in the target device area within a preset time period, and it is determined that the operation plan data represents that the operation plan in the target device area within the preset time period is not in the execution state, then generate a second warning message; wherein, the second warning message represents that there is an unauthorized operation in the target device area within the preset time period.

[0195] Exemplarily, the device compares each recognition result with the operation plan data. If it is determined that all recognition results represent the presence of operating personnel in the target device area within a preset time period, and it is determined that the operation plan data represents that there is an operation plan in the target device area within the preset time period and the operation plan is not in the execution state, then generate a second warning message to represent that there is an unauthorized operation in the target device area within the preset time period.

[0196] For example, compare the two pieces of data "There are operating personnel in a certain equipment area of a certain substation at a certain time point" and "The operation plan data of the equipment area of the substation at the time point". If there are operating personnel but no corresponding operation plan, push a warning message of "out-of-cycle operation" through voice, text, sound and light, etc.; if there are operating personnel with a corresponding operation plan but the operation plan is not in the "execution" state, push a warning message of "unauthorized operation"; if there are operating personnel with a corresponding operation plan and the operation plan is in the "execution" state, it means that the operating personnel are operating according to the plan and no warning message needs to be pushed.

[0197] Among them, based on the hierarchical early warning processing logic, the specific process is as follows:

[0198] def handle_alert(camera_info, plan_status):

[0199] msg_template = {

[0200] "UNAUTHORIZED": {

[0201] "level": "Emergency",

[0202] "content": f"{camera_info['substation']} {camera_info['area']} has unauthorized operations"

[0203] },

[0204] "NO_PLAN": {

[0205] "level": "Severe",

[0206] "content": f"{camera_info['substation']} {camera_info['area']} has extracorporeal circulation operations"

[0207] }

[0208] }

[0209] if plan_status in msg_template:

[0210] send_alert(

[0211] level=msg_template[plan_status]['level'],

[0212] message=msg_template[plan_status]['content'] )

[0214] log_operation(camera_info, plan_status) # Record the audit log.

[0215] Taking another example, for the identification process of extracorporeal circulation of operations, the corresponding system architecture is as follows:

[0216] graph TD

[0217] A[Multi-camera Manager] --> B[Parallel Screenshot]

[0218] B --> C[Human Detection Cluster]

[0219] C --> D{Is there anyone present?}

[0220] D -->|Yes| E[Ledger Information Retrieval]

[0221] D -->|No| A

[0222] E --> F[Job Plan Status Verification]

[0223] F --> G{Is the status compliant?}

[0224] G -->|No| H[Hierarchical Early Warning Push]

[0225] G -->|Yes| A

[0226] Among them, for the identification process of job extracorporeal circulation, the corresponding main logic integration process is as follows:

[0227] from concurrent.futures import ThreadPoolExecutor

[0228] def process_camera(camera_id):

[0229] try:

[0230] # Step 1: Obtain video screenshot

[0231] img_path = capture_frame(camera_id)

[0232] # Step 2: Human detection (triple verification)

[0233] if not multi_check_human(img_path):

[0234] return

[0235] # Step 3: Obtain ledger information

[0236] camera_info = get_camera_info(camera_id)

[0237] if not camera_info:

[0238] return

[0239] # Step 4: Verify the job plan

[0240] plan_status = check_work_plan_status(

[0241] camera_info['substation'],

[0242] camera_info['area'] )

[0244] # Step 5: Trigger an alert

[0245] if plan_status != "IN_PROGRESS":

[0246] handle_alert(camera_info, plan_status)

[0247] except Exception as e:

[0248] log.error(f"Exception occurred while processing camera {camera_id}: {str(e)}")

[0249] Among them, an example of the alert information for unauthorized operation can be:

Intelligent Operation Monitoring System

[0250] It is worth adding that in order to further improve the recognition accuracy of out-of-cycle operations, monitoring indicators can be set to perform the recognition task of out-of-cycle operations. For example, the monitoring indicators can include: the number of screenshots processed daily, recognition accuracy rate (requiring regular annotation of the test set), timely alert response rate, single-camera processing delay (<3 seconds), support for concurrent analysis of 200 cameras, system availability (>99.95%); environmental risk alerts can also be achieved by combining weather forecasts, and a mobile terminal disposal confirmation function can be developed. In order to further improve the recognition security of out-of-cycle operations, security measures such as accessing the video private network, using a whitelist IP for crawlers, and encrypting email push can be taken.

[0251] In this embodiment, based on the above embodiment, on the one hand, face recognition / work uniform recognition is used to verify the identity of the operator, further improving the recognition accuracy of the extracorporeal circulation of the operation. On the other hand, based on the intelligent false alarm filtering mechanism, if it is determined that there are staff members in the target equipment area within a preset time period and the number of occurrences exceeds the preset number, it is determined that there are operators in the target equipment area, which can also further improve the recognition accuracy of the extracorporeal circulation of the operation.

[0252] Figure 3 FIG. is a schematic structural diagram of an identification device for extracorporeal circulation of an operation provided by an embodiment of the present application. As Figure 3 shown, the device includes:

[0253] An identification module 401, configured to obtain video stream data collected by an image acquisition device; and perform identification processing on the video stream data to obtain at least one identification result corresponding to the video stream data; wherein, the identification result includes whether there are operators in the target equipment area in the target substation within a preset time period.

[0254] An acquisition module 402, configured to obtain operation plan data corresponding to the target equipment area within a preset time period; wherein, the operation plan data represents the operation plan situation of the target equipment area.

[0255] An early warning module 403, configured to generate a first early warning information if it is determined that each identification result indicates that there are operators in the target equipment area within a preset time period and the operation plan data indicates that there is no operation plan in the target equipment area within the preset time period; wherein, the first early warning information represents that there is extracorporeal circulation of the operation in the target equipment area at the corresponding moment.

[0256] In a possible implementation manner, the identification module 401 is specifically configured to: perform an i-th solution processing on the video stream data to obtain an i-th image set of the video stream data; wherein, the image set includes operation scene images of at least one target equipment area; perform image recognition processing on each operation scene image in the i-th image set to obtain an i-th result set; wherein, the i-th result set includes a processing result corresponding to each operation scene image; the processing result represents whether an operator appears in the target equipment area at the corresponding moment; and determine the i-th identification result according to the i-th result set.

[0257] In a possible implementation manner, the identification module 401 is specifically configured to: perform face recognition processing on the operation scene image to obtain a processing result corresponding to the operation scene image; or perform work uniform recognition processing on the operation scene image to obtain a processing result corresponding to the operation scene image.

[0258] In a possible implementation manner, the recognition module 401 is further specifically configured to: determine the total number of target operation scenario images corresponding to the i-th image set according to each processing result in the i-th result set; wherein, the target operation scenario image is an operation scenario image in which a staff member appears in the target device area at the corresponding moment; if it is determined that the total number is greater than a preset number, it is determined that the i-th recognition result represents that there are operation personnel in the target device area within a preset time period.

[0259] In a possible implementation manner, before the recognition module 401 is used to obtain the video stream data collected by the image acquisition device, the device is further configured to: obtain the operation plan data of the target device area at at least one moment; if it is determined that the operation plan data represents that there is no operation plan in the target device area at the corresponding moment, or if it is determined that the operation plan data represents that the operation plan in the target device area at the corresponding moment is not in an executing state, then execute the step of obtaining the video stream data collected by the image acquisition device.

[0260] In a possible implementation manner, the device is further configured to: if it is determined that each recognition result represents that there are operation personnel in the target device area within a preset time period, and it is determined that the operation plan data represents that the operation plan in the target device area within the preset time period is not in an executing state, then generate a second warning message; wherein, the second warning message represents that there is an unauthorized operation situation in the target device area within the preset time period.

[0261] In a possible implementation manner, the device is further configured to: determine the health status of the video stream data according to the status information of the image acquisition device; wherein, the status information represents whether the image acquisition device is in an abnormal state; the abnormal state is an offline state or a frame rate abnormal state.

[0262] The device in this embodiment can execute the technical solutions in the above method, and the specific implementation process and technical principle are the same, which will not be elaborated here.

[0263] Figure 4 The structure diagram of an electronic device provided by an embodiment of the present application is shown as Figure 4 shown, the electronic device includes: a memory 501, a processor 502; the memory 501 is a memory for storing executable instructions of the processor 502.

[0264] Wherein, the processor 502 is configured to execute the method provided in the above embodiment.

[0265] The electronic device further includes a receiver 503 and a transmitter 504. The receiver 503 is used to receive instructions and data sent by other devices, and the transmitter 504 is used to send instructions and data to external devices.

[0266] For the specific implementation process of the processor, reference may be made to the above method embodiments. Their implementation principles and technical effects are similar, and will not be elaborated here in this embodiment.

[0267] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU for short), or other general-purpose processors, digital signal processors (DSP for short), application specific integrated circuits (ASIC for short), etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0268] The embodiment of the present application also provides a chip for running instructions, and this chip is used to execute the technical solutions in the above embodiments.

[0269] The embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions run on a computer, the computer is enabled to execute the technical solutions in the above embodiments.

[0270] The above-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium may be any available medium accessible by a general or special-purpose computer.

[0271] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium may also be a component of the processor. The processor and the readable storage medium may be located in an application specific integrated circuit. Of course, the processor and the readable storage medium may also exist as discrete components in a device.

[0272] The embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when at least one processor executes the computer program, the technical solutions in the above embodiments can be implemented.

[0273] The division of units is merely a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed among each other can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0274] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0275] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0276] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present invention. And the aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical discs and other various media that can store program codes.

[0277] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When this program is executed, it executes the steps including the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disks or optical discs and other various media that can store program codes.

[0278] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include well-known common general knowledge or conventional technical means in the technical field not disclosed in the present invention. It is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A method for identifying extracorporeal circulation, characterized in that: include: Acquire video stream data collected by an image acquisition device; and performing recognition processing on the video stream data to obtain at least one recognition result corresponding to the video stream data; wherein the recognition result includes whether there is an operating personnel in the target equipment area in the target substation within a preset time period; Acquire operation plan data corresponding to the target device area within a preset time period; wherein the operation plan data represents the operation plan status of the target device area; If it is determined that each of the identification results indicates that there are operating personnel in the target equipment area within the preset time period, and the operation plan data indicates that there is no operation plan in the target equipment area within the preset time period, a first warning information is generated; wherein, the first warning information indicates that there is extracorporeal circulation in the target equipment area at the corresponding moment.

2. The method according to claim 1, characterized in that Performing recognition processing on the video stream data to obtain at least one recognition result corresponding to the video stream data includes: Performing an i-th calculation process on the video stream data to obtain an i-th image set of the video stream data; wherein the image set includes at least one operating scene image of the target device area; Performing image recognition processing on each work scene image in the i-th image set to obtain an i-th result set; wherein the i-th result set includes a processing result corresponding to each of the work scene images; the processing result represents whether an operator appears in the target equipment area at the corresponding moment; According to the i-th result set, the i-th recognition result is determined.

3. The method according to claim 2, characterized in that Perform image recognition processing on each work scene image in the i-th image set to obtain the i-th result set, including: Performing face recognition processing on the operation scene image to obtain a processing result corresponding to the operation scene image; Alternatively, work clothes recognition processing is performed on the work scene image to obtain a processing result corresponding to the work scene image.

4. The method according to claim 2, characterized in that: According to the i-th result set, determining the i-th recognition result includes: According to each processing result in the i-th result set, determine the total number of target work scene images corresponding to the i-th image set; wherein the target work scene image is a work scene image in which a worker appears in the target equipment area at the corresponding moment represented by the processing result; If it is determined that the total number is greater than the preset number, it is determined that the i-th recognition result indicates that there is an operator in the target equipment area within the preset time period.

5. The method according to claim 1, characterized in that: Before acquiring the video stream data acquired by the image acquisition device, the method further includes: Acquire operation plan data of the target equipment area at at least one time; If it is determined that the job plan data indicates that there is no job plan for the target device area at the corresponding moment, or if it is determined that the job plan data indicates that the job plan for the target device area at the corresponding moment is not in execution, the step of obtaining the video stream data captured by the image acquisition device is executed.

6. The method according to claim 1, characterized in that The method further comprises: If it is determined that each of the identification results indicates that there are operating personnel in the target equipment area within the preset time period, and it is determined that the operation plan data indicates that the operation plan for the target equipment area within the preset time period is not in execution, a second warning information is generated; wherein, the second warning information indicates that there is unauthorized operation in the target equipment area within the preset time period.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: The health status of the video stream data is determined according to the status information of the image acquisition device; wherein the status information indicates whether the image acquisition device is in an abnormal state; the abnormal state is an offline state or an abnormal frame rate state.

8. An identification device for extracorporeal circulation, characterized in that: include: An identification module, used to obtain video stream data collected by an image acquisition device; and performing recognition processing on the video stream data to obtain at least one recognition result corresponding to the video stream data; wherein the recognition result includes whether there is an operating personnel in the target equipment area in the target substation within a preset time period; An acquisition module, used to acquire operation plan data corresponding to the target device area within a preset time period; wherein the operation plan data represents the operation plan status of the target device area; The early warning module is used to generate a first early warning message if it is determined that each of the identification results indicates that there are operating personnel in the target equipment area within a preset time period, and the operation plan data indicates that there is no operation plan in the target equipment area within the preset time period; wherein the first early warning message indicates that there is extracorporeal circulation in the target equipment area at the corresponding moment.

9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

11. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images