Risk detection method

Abstract

The embodiment of the specification provides a risk detection method, wherein the risk detection method comprises the following steps: acquiring a monitoring video collected by a video collection device for an operating position in a coal mine underground, and extracting a target video frame from the monitoring video; positioning a risk area in the target video frame according to the target video frame when the operating position has a risk situation, and determining an image collection parameter corresponding to the risk area; collecting a risk image corresponding to the risk area by an image collection device arranged at the operating position according to the image collection parameter; inputting the risk image into a risk detection model for processing, obtaining a risk level of the risk area, and performing an alarm task corresponding to the risk area according to the risk level.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of image processing, and in particular to a risk detection method. BACKGROUND

[0002] With the development of computer and Internet technology, the traditional coal industry begins to digitize and transform. By integrating key technologies such as Internet of Things, cloud computing, big data, and artificial intelligence, breakthroughs are made in core technologies such as Internet of Things access, intelligent data analysis, three-dimensional visualization, and production, transportation, and sales collaborative management. An intelligent comprehensive management and control platform for coal mines is built, which enables the traditional coal industry to move away from the original management and control mode and enter the digital management and control mode, effectively promoting the intelligent development of the coal industry and facilitating the construction of intelligent coal mines. In the prior art, in order to ensure the safety of the operation of underground personnel in a coal mine, monitoring devices are usually deployed at the operating position to monitor the safety of the operator in real time. In the event of a safety risk, the supervisor can be reminded in a timely manner, or an alarm device can be used to alert the worker to ensure the safety of the worker. However, real-time monitoring by monitoring devices not only consumes human resources, but also frequently triggers false alarms, which seriously affects the normal progress of work. Therefore, an effective solution is urgently needed to solve the above problems. SUMMARY

[0003] Therefore, the embodiments of the present specification provide a risk detection method. One or more embodiments of the present specification also relate to a risk detection device, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects in the prior art.

[0004] According to a first aspect of the embodiments of the present specification, a risk detection method is provided, comprising:

[0005] acquiring a monitoring video collected by a video collection device for an operating position in a coal mine, and extracting a target video frame from the monitoring video;

[0006] According to the target video frame, if the operating position has a risk, a risk area is located in the target video frame, and an image collection parameter corresponding to the risk area is determined;

[0007] An image collection device deployed at the operating position collects a risk image corresponding to the risk area according to the image collection parameter;

[0008] The risk image is input into a risk detection model for processing to obtain a risk level of the risk area, and an alarm task corresponding to the risk area is performed according to the risk level.

[0009] According to a second aspect of the embodiments of this specification, a risk detection device is provided, comprising:

[0010] The acquisition module is configured to acquire monitoring videos collected by video acquisition devices at underground operation locations in coal mines, and extract target video frames from the monitoring videos.

[0011] The determination module is configured to locate a risk region in the target video frame and determine the image acquisition parameters corresponding to the risk region if the operation position is identified as having a risk based on the target video frame.

[0012] The acquisition module is configured to acquire risk images corresponding to the risk area according to the image acquisition parameters, through an image acquisition device deployed at the operation location.

[0013] The processing module is configured to input the risk image into the risk detection model for processing, obtain the risk level of the risk area, and execute the alarm task corresponding to the risk area according to the risk level.

[0014] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising:

[0015] Memory and processor;

[0016] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the risk detection method described above.

[0017] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed by a processor, implement the steps of the risk detection method described above.

[0018] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the steps of the risk detection method described above.

[0019] The risk detection method provided in this embodiment aims to improve the accuracy of risk detection in underground coal mines. After acquiring monitoring video of the operation location in the mine using video acquisition equipment, risk detection can be completed based on the monitoring video through subsequent automated processing. In this process, target video frames can be extracted from the monitoring video. If a risk is initially identified at the operation location based on the target video frames, the risk area can be located in the target video frames, and the corresponding image acquisition parameters can be determined. Then, a high-precision image acquisition device can be used to acquire the risk image corresponding to the risk area according to the image acquisition parameters. After acquiring a high-precision risk image for the risk area, the risk image can be input into the risk detection model for processing to obtain the risk level of the risk area. This enables the model to quickly and accurately determine the risk level of the risk area, allowing for the subsequent execution of alarm tasks corresponding to the risk level. This risk detection architecture, combining image acquisition equipment with video acquisition equipment, effectively improves the accuracy of risk detection. Furthermore, by combining it with a machine learning model for risk prediction, the speed of risk detection can be improved, thereby ensuring the safety of underground coal mine operators. Attached Figure Description

[0020] Figure 1 This is a flowchart of a risk detection method provided in one embodiment of this specification;

[0021] Figure 2 This is a schematic diagram of a risk image in a risk detection method provided in one embodiment of this specification;

[0022] Figure 3 This is a flowchart illustrating the processing procedure of a risk detection method provided in one embodiment of this specification;

[0023] Figure 4 This is a schematic diagram of the structure of a risk detection device provided in one embodiment of this specification;

[0024] Figure 5 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0025] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0026] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0027] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0028] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0029] This specification provides a risk detection method, and also relates to a risk detection device, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0030] See Figure 1 , Figure 1 A flowchart of a risk detection method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0031] Step S102: Acquire the monitoring video collected by the video acquisition device for the underground operation position in the coal mine, and extract the target video frame from the monitoring video.

[0032] The risk detection method provided in this embodiment is applied to the safety detection scenario of the working area of ​​underground workers in coal mines. It is used to complete accurate risk detection by cooperating with video acquisition equipment and image acquisition equipment, so as to ensure the safe work of workers and avoid the interruption of the process caused by false alarms.

[0033] Specifically, the video acquisition equipment refers to cameras deployed at the operating positions in underground coal mines. These cameras monitor the operating positions in real time, allowing for accurate risk detection results when triggered, thus improving worker safety. The operating position refers to any location in the coal mine where workers need to participate in the work, such as the material loading area of ​​a conveyor belt. The monitoring video refers to video segments captured in real time by the video acquisition equipment at the operating position. The target video frame refers to the video frame used for preliminary risk detection within the monitoring video. This frame can be a randomly selected frame from the monitoring video or a frame selected using a pre-defined algorithm, such as based on resolution. In this embodiment, no specific limitations are imposed on the implementation.

[0034] Therefore, to improve the accuracy of risk detection in coal mines, after acquiring monitoring videos of underground operating locations using video acquisition equipment, risk detection can be completed through subsequent automated processing based on the monitoring videos. In this process, target video frames can be extracted from the monitoring videos. If a risk is initially identified at the operating location based on the target video frames, the risk area can be located within the target video frames, and the corresponding image acquisition parameters can be determined. Then, a high-precision image acquisition device can be used to acquire risk images corresponding to the risk area according to the image acquisition parameters. After acquiring high-precision risk images for the risk area, these images can be input into a risk detection model for processing to obtain the risk level of the risk area. This allows for rapid prediction using the model, accurately and quickly determining the risk level of the risk area, and subsequently executing the corresponding alarm tasks based on the risk level. This risk detection architecture, combining image acquisition equipment with video acquisition equipment, can effectively improve risk detection accuracy. Furthermore, combining it with a machine learning model for risk prediction can improve risk detection speed, thereby ensuring the safety of underground coal mine operators.

[0035] In one or more embodiments of this example, extracting the target video frame from the surveillance video includes:

[0036] The monitoring video is processed by frame segmentation to obtain a video frame sequence. Video frames with a resolution greater than a resolution threshold are selected from the video frame sequence as target video frames. The step of locating a risk area in the target video frame when the operation location is identified as having a risk based on the target video frame includes: inputting the target video frame into a risk classification model for processing, and performing the step of locating the risk area in the target video frame when the processing result determines that the operation location has a risk.

[0037] Specifically, clarity refers to the image clarity of each video frame in the video frame sequence, and risk classification model refers to a binary classification model used to initially detect whether there is risky behavior at the operation location. Its input is the target video frame, and its output is the probability that there is risky behavior in the target video frame.

[0038] Based on this, after acquiring monitoring video of the underground operation location in the coal mine using video acquisition equipment, the monitoring video can be processed into frames to obtain a video frame sequence. Then, video frames with a resolution greater than a resolution threshold can be selected from this sequence as target video frames. If there are many video frames with a resolution greater than the resolution threshold, one can be randomly selected as the target video frame. After obtaining the target video frame, it can be input into a risk classification model for processing. If the processing result determines that the operation location poses a risk, the step of locating the risk area within the target video frame is executed.

[0039] In practice, in addition to filtering target video frames in the above way, target video frames can also be filtered by comparing the brightness value of each video frame in the video frame sequence. By selecting the video frame with the highest brightness value as the target video frame, the image clarity can be guaranteed, thereby improving the accuracy of the initial risk detection.

[0040] For example, at the loading position of a conveyor belt in an underground coal mine, operators need to stack coal and mineral materials onto the conveyor. Because the conveyor belt operates at high speed and generates significant driving force, operators must maintain a safe distance from it. In this situation, a camera deployed at the loading position will constantly monitor the operator's safety, preventing dangerous situations such as the operator being dragged by the conveyor belt. After the monitor acquires the video, it can perform frame segmentation to obtain a sequence of n video frames. Then, by selecting the video frame with the highest resolution from the sequence, the selected video frame is shown below. Figure 2 As shown in (a), the video frame can then be input into the risk classification model for preliminary risk detection. If the detection results indicate that there is a risk at the current feeding position, subsequent in-depth risk detection processing can be triggered.

[0041] In summary, by selecting high-resolution video frames as target video frames, the detection accuracy can be improved in the initial risk detection stage, which in turn facilitates subsequent in-depth risk detection.

[0042] Step S104: If the operation location is identified as having a risk based on the target video frame, locate the risk area in the target video frame and determine the image acquisition parameters corresponding to the risk area.

[0043] Specifically, after obtaining the target video frame, a preliminary risk detection can be performed on the operation location based on the target video frame. If the detection results indicate that the operation location is at risk, it means that the operation location may be at risk. To ensure the accuracy of risk detection, a secondary risk detection processing logic can be triggered through image acquisition equipment. In this process, the risk area can first be located in the target video frame to determine the risk area that triggers the risk alarm through image localization. Then, the image acquisition parameters corresponding to the risk area can be determined so that the image acquisition equipment can subsequently acquire images of the risk area according to the image acquisition parameters, obtaining sufficiently clear images that correspond only to the risk area for risk analysis, thereby improving the accuracy of risk detection.

[0044] Specifically, the risk area refers to the image region in the target video frame that triggers an initial risk detection result. This allows subsequent image acquisition devices to focus solely on this region, avoiding interference from excessive redundant information during image acquisition and slowing down subsequent risk detection. Correspondingly, the image acquisition parameters refer to the parameters that the image acquisition device needs to adjust when acquiring images of the risk area. These include adjusting the focal length, deflection angle, and exposure of the image acquisition device to ensure more accurate images. This process is primarily due to the fact that video acquisition devices monitor the entire area of ​​the operation location to achieve global monitoring, which inevitably leads to low video quality. Deploying a separate image acquisition device and acquiring images according to the specified parameters effectively avoids this problem, resulting in clearer images and significantly improving the accuracy of subsequent risk detection.

[0045] In one or more embodiments of this example, determining the image acquisition parameters corresponding to the risk area includes:

[0046] When the video acquisition device and the image acquisition device are located in the same location, the video acquisition parameters corresponding to the video acquisition device are obtained, and the first region information corresponding to the risk area is determined. The image acquisition parameters corresponding to the risk area are then constructed based on the video acquisition parameters and the first region information. Alternatively, when the video acquisition device and the image acquisition device are not located in the same location, the video acquisition parameters corresponding to the video acquisition device are transformed according to a preset parameter transformation matrix to obtain candidate video acquisition parameters. The second region information corresponding to the risk area is then determined, and the image acquisition parameters corresponding to the risk area are then constructed based on the candidate video acquisition parameters and the second region information.

[0047] Specifically, video acquisition parameters refer to the acquisition parameters corresponding to the video acquisition device when acquiring surveillance video, such as the focal length, deflection angle, and exposure of the video acquisition device. First region information and second region information specifically refer to the distance information of the risk area in the target video frame relative to the video acquisition device. The parameter transformation matrix is ​​a transformation matrix constructed based on the coordinates of the image acquisition device and the video acquisition device in the world coordinate system. During image acquisition, this matrix is ​​used to map the video acquisition parameters corresponding to the video acquisition device to the image acquisition device, and then construct the image acquisition parameters.

[0048] Therefore, when it is necessary to acquire risk images using an image acquisition device, if the video acquisition device and the image acquisition device are located in the same position, the image acquisition device can directly reuse the video acquisition parameters corresponding to the video acquisition device to adjust its own image acquisition parameters. Therefore, the video acquisition parameters corresponding to the video acquisition device can be obtained first. At the same time, in order to ensure that the image acquisition device can acquire images corresponding to the risk area, the first area information corresponding to the risk area can be determined simultaneously. Based on this, the image acquisition parameters corresponding to the risk area can be constructed according to the video acquisition parameters and the first area information, so that the image acquisition device can subsequently acquire risk images of the risk area according to these parameters.

[0049] If the video acquisition device and the image acquisition device are not located in the same place, it means that the image acquisition device cannot reuse the parameters of the video acquisition device for subsequent image acquisition. Therefore, a pre-constructed parameter transformation matrix can be determined. Then, the video acquisition parameters corresponding to the video acquisition device can be transformed according to the preset parameter transformation matrix to obtain candidate video acquisition parameters. At the same time, the second region information corresponding to the risk area can be determined. Based on this, the image acquisition parameters corresponding to the risk area can be constructed according to the candidate video acquisition parameters and the second region information, so that the subsequent image acquisition device can perform risk image acquisition on the risk area according to these parameters.

[0050] In practice, considering the potential for false alarms after video capture devices capture video of the operation location, when deploying image capture devices at the operation location, it is advisable to deploy them on the opposite side of the video capture devices. This allows the image capture devices to capture images of the risk area from other angles, thereby making subsequent risk detection more accurate.

[0051] Continuing with the previous example, if the camera is deployed on wall A corresponding to the loading position, it can be deployed on wall B corresponding to the loading position. Wall B can be the wall opposite to wall A. Then, information such as the camera's focal length and deflection angle can be obtained, and the risk area in the target video frame can be determined. Next, the camera's focal length and deflection angle can be converted into the corresponding camera coordinates using a preset transformation matrix. Then, combining the converted focal length and deflection angle with the area information corresponding to the risk area, the camera's focal length and deflection angle can be calculated. This allows for the subsequent acquisition of risk images corresponding to the risk area using the camera for deep risk detection.

[0052] In summary, before acquiring risk images using image acquisition devices, to improve the accuracy of risk image acquisition, image acquisition parameters can be constructed based on the location of the image acquisition device. This allows the image acquisition device to acquire risk images of the corresponding locations of risk areas according to the image acquisition parameters, ensuring that the risk images can more intuitively reflect risky behaviors.

[0053] Step S106: Using the image acquisition device deployed at the operation location, acquire the risk image corresponding to the risk area according to the image acquisition parameters.

[0054] Specifically, after obtaining the image acquisition parameters corresponding to the risk area, the image acquisition device deployed at the operation location can then acquire images of the risk area. In order to ensure the accuracy of image acquisition, the image acquisition device can acquire risk images corresponding to the risk area according to the image acquisition parameters, so that subsequent detection of whether risky behavior has occurred at the operation location can be completed based on the risk images.

[0055] Specifically, a risk image refers to an image acquired by an image acquisition device for a risk area in a target video frame. It can be one or more images, which are then input into a risk detection model for risk detection processing at the operation location to ensure the accuracy of risk detection.

[0056] In one or more embodiments of this example, the image acquisition device deployed at the operating location acquires a risk image corresponding to the risk area according to the image acquisition parameters, including:

[0057] The image acquisition device deployed at the operation location acquires an initial risk image corresponding to the risk area according to the image acquisition parameters; it detects whether the initial risk image meets the risk detection conditions; if so, the initial risk image is used as the risk image corresponding to the risk area; if not, candidate video frames are extracted from the monitoring video, and the candidate video frames are used as the target video frames. The step of locating the risk area in the target video frames is performed until a risk image that meets the risk detection conditions is acquired.

[0058] Specifically, the initial risk image refers to a risk image acquired by the image acquisition device that has not yet undergone usability testing. Risk detection conditions refer to the conditions used to determine whether the initial risk image can be used for risk detection, such as whether the initial risk image meets the set resolution or whether it is clear. Candidate video frames refer to reselected video frames used to re-acquire risk images.

[0059] Based on this, after determining the image acquisition parameters, the image acquisition device deployed at the operational location can acquire the initial risk image corresponding to the risk area according to the image acquisition parameters. Then, it can be checked whether the initial risk image meets the risk detection conditions. If so, the initial risk image can be used as the risk image corresponding to the risk area for subsequent risk detection processing. If not, it indicates that the acquired risk image is unusable or ineffective, further suggesting that the image acquisition parameters set by the image acquisition device are unreasonable. Therefore, the image acquisition parameters need to be adjusted. To save time, candidate video frames can be extracted again from the surveillance video, and these candidate video frames can be used as the target video frames. The step of locating the risk area in the target video frames is then performed until a risk image meeting the risk detection conditions is acquired at a certain time, or a set time threshold is reached. The image with the highest clarity from multiple initial risk images is then selected as the risk image to improve the efficiency of subsequent risk detection.

[0060] In practical applications, in addition to the methods described above for screening risk images, risk regions in the target video frame can be cropped to obtain associated images. Then, the similarity between the associated image and the initial risk image can be calculated. If the similarity is greater than a preset similarity threshold, it can be used as a risk image; otherwise, parameters are adjusted and new risk images are acquired. The similarity can be determined using cosine similarity calculations.

[0061] In summary, performing detection on the acquired images enables the subsequent use of high-precision and highly accurate risk images for risk detection, thereby improving the accuracy of risk detection in underground coal mines.

[0062] In one or more embodiments of this example, before the step of inputting the risk image into a risk detection model for processing to obtain the risk level of the risk area is executed, the method further includes:

[0063] A reference image that has a time-aligned relationship with the risk image is identified; an image restoration feature corresponding to the risk image is constructed based on the reference image; the risk image is updated using the image restoration feature to obtain a target risk image; the target risk image is used as the risk image, and the risk image is input into a risk detection model for processing to obtain the risk level of the risk region.

[0064] Specifically, the reference image refers to an image selected from images acquired by the image acquisition device that has a time alignment relationship with the risk image. This time alignment relationship specifically means that the images were acquired at the same time as the risk image or at a time 20-24 hours apart. Correspondingly, the image inpainting feature refers to the features used to inpaint the risk image. It can be understood as the filter values ​​used to update the pixel values ​​in the risk image, thereby updating the risk image to the target risk image and eliminating errors caused by environmental influences during image acquisition.

[0065] Therefore, in order to ensure that the risk image accurately represents the true situation of the risk area, alignment and restoration can be performed after obtaining the risk image to eliminate inaccuracies caused by dim lighting or dust. In this process, a reference image with a time-aligned relationship to the risk image can be identified. Then, image restoration features corresponding to the risk image can be constructed based on the reference image. This process can be understood as calculating the average pixel value of the reference image, using the calculation result as the image restoration feature, and then using the image restoration feature to update the risk image to obtain the target risk image. Finally, the target risk image is used as the actual risk image, and the risk image is input into a risk detection model for processing to obtain the risk level of the risk area.

[0066] In practical applications, when constructing image restoration features, the risk image can be determined by subtracting the average pixel value from the reference image. Then, the calculated result is added to the pixel value corresponding to each pixel in the risk image to obtain the target risk image for subsequent risk detection processing.

[0067] Following the previous example, after obtaining the image acquisition parameters for the camera, the camera can be controlled to acquire images of the risk area. If the acquired image is not clear enough, new images can be acquired again until a clearer risk image is obtained. Then, the risk image can be repaired to eliminate the influence of the underground coal mine environment, thus obtaining the target risk image.Figure 2 As shown in (b), the risk detection model can then be used to detect the risk and determine the risk detection result.

[0068] In summary, considering that the dark environment or dust in underground coal mines may lead to unclear images, the collected images can be repaired to improve image clarity, thereby enhancing the accuracy of subsequent risk detection.

[0069] Step S108: Input the risk image into the risk detection model for processing to obtain the risk level of the risk area, and execute the alarm task corresponding to the risk area according to the risk level.

[0070] Specifically, after obtaining the risk image captured by the image acquisition device for the risk area, in order to improve the accuracy and efficiency of risk detection, the risk image can be input into the risk detection model for processing to obtain the risk level of the risk area, so that the alarm task corresponding to the risk area can be executed according to the risk level.

[0071] Specifically, the risk detection model refers to a machine learning model that takes a risk image as input and outputs a risk level. Correspondingly, the risk level refers to the risk rating output by the risk model for a risky area; a higher rating indicates a greater risk. A level 0 risk rating indicates that no risk exists in the risky area. The alarm task refers to the alarm methods set for different risk levels when a risk exists in the risky area. For example, a level 1 risk might trigger a voice broadcast, a level 2 risk a warning light, a level 3 risk a partial power outage, and a level 4 risk a complete power outage of the entire mine area. This embodiment does not impose any limitations.

[0072] In one or more embodiments of this example, executing the alarm task corresponding to the risk area according to the risk level includes:

[0073] When the risk level is Level 1, the alert and alarm task corresponding to the risk area is executed according to the risk level; when the risk level is Level 2, the prompt and alarm task corresponding to the risk area is executed according to the risk level; when the risk level is Level 3, the power outage alarm task corresponding to the risk area is executed according to the risk level.

[0074] Specifically, the alert / alarm task refers to issuing an alert to the operator regarding a risky behavior, such as through voice broadcast or sending instructions to management personnel for reminder. The warning / alarm task refers to issuing a warning to the operator through device settings, such as using a brightly colored light at the operating location. The power outage alarm task refers to issuing an alert regarding a power outage at the operating location.

[0075] Based on this, after obtaining the risk level output by the model, different alarm tasks can be executed for different risk levels. When the risk level is Level 1, the reminder alarm task corresponding to the risk area can be executed according to the risk level. When the risk level is Level 2, the prompt alarm task corresponding to the risk area can be executed according to the risk level. When the risk level is Level 3, the power outage alarm task corresponding to the risk area can be executed according to the risk level.

[0076] Continuing with the previous example, when collecting images through a camera, such as... Figure 2 After the risk image shown in (b), the risk image can be input into the risk detection model for processing. If the model outputs a level 1 risk, the manager can verbally remind the operator. If the model outputs a level 2 risk, the alarm light at the material loading position can be used to remind the operator. If the model outputs a level 3 risk, the belt conveyor can be shut down to avoid affecting the personal safety of the operator.

[0077] In summary, selecting different alarm tasks for different risk levels can effectively manage risks, thereby improving the safety of staff while avoiding disruption to work.

[0078] In one or more embodiments of this example, the training of the risk detection model includes:

[0079] Obtain sample risk images and their corresponding sample risk levels; construct sample expansion prompts based on the sample risk images and their risk levels, and input the sample expansion prompts into a large language model; use the large language model to recall expanded risk images from an external database according to the sample expansion prompts, and generate expanded risk levels corresponding to the expanded risk images; train an initial risk detection model based on the expanded risk images and their risk levels until a risk detection model that meets the training stopping condition is obtained.

[0080] Specifically, "sample risk image" refers to the image used to train the risk detection model, and "sample risk level" is the sample label used for model training. "Sample expansion prompt words" refers to prompt words retrieved from an external database using a large language model to identify images similar to the associated sample risk images. The external database refers to a third-party database used to store images related to coal mine risks. Correspondingly, "expanded risk image" refers to images retrieved using a large language model that can be used to train the risk detection model. The corresponding expanded risk level can be labeled using the large language model.

[0081] Based on this, in order to achieve high prediction accuracy for the risk detection model, sample risk images and their corresponding risk levels can be obtained. Subsequently, sample expansion prompts can be constructed based on the sample risk images and risk levels, and these prompts are input into a large language model. The large language model then retrieves expanded risk images from an external database according to the expanded prompts and generates expanded risk levels corresponding to the expanded risk images. This expands the sample set. Afterward, the initial risk detection model can be trained based on the expanded risk images and risk levels until a risk detection model that meets the training stopping condition is obtained.

[0082] In practical applications, training stopping conditions include, but are not limited to, loss value comparison conditions, iteration count conditions, or validation set verification conditions. In specific implementation, these conditions can be set according to actual needs, and this embodiment does not impose any limitations on them.

[0083] The risk detection method provided in this embodiment aims to improve the accuracy of risk detection in underground coal mines. After acquiring monitoring video of the operation location in the mine using video acquisition equipment, risk detection can be completed based on the monitoring video through subsequent automated processing. In this process, target video frames can be extracted from the monitoring video. If a risk is initially identified at the operation location based on the target video frames, the risk area can be located in the target video frames, and the corresponding image acquisition parameters can be determined. Then, a high-precision image acquisition device can be used to acquire the risk image corresponding to the risk area according to the image acquisition parameters. After acquiring a high-precision risk image for the risk area, the risk image can be input into the risk detection model for processing to obtain the risk level of the risk area. This enables the model to quickly and accurately determine the risk level of the risk area, allowing for the subsequent execution of alarm tasks corresponding to the risk level. This risk detection architecture, combining image acquisition equipment with video acquisition equipment, effectively improves the accuracy of risk detection. Furthermore, by combining it with a machine learning model for risk prediction, the speed of risk detection can be improved, thereby ensuring the safety of underground coal mine operators.

[0084] The following is in conjunction with the appendix Figure 3 Taking the application of the risk detection method provided in this specification in a coal mine underground safety detection scenario as an example, the risk detection method will be further explained. Among other things, Figure 3 The present specification shows a flowchart of a risk detection method according to an embodiment, which includes the following steps.

[0085] Step S302: Acquire monitoring video collected by the video acquisition device at the underground operation location in the coal mine.

[0086] Step S304: Perform frame segmentation on the monitoring video to obtain a video frame sequence. Select video frames with a resolution greater than a resolution threshold from the video frame sequence as target video frames.

[0087] Step S306: Input the target video frame into the risk classification model for processing. If the processing result determines that there is a risk at the operation location, locate the risk area in the target video frame.

[0088] Step S308: When the video acquisition device and the image acquisition device are located in the same position, obtain the video acquisition parameters corresponding to the video acquisition device and determine the first area information corresponding to the risk area, and construct the image acquisition parameters corresponding to the risk area based on the video acquisition parameters and the first area information.

[0089] Step S310: When the video acquisition device and the image acquisition device are not in the same location, the video acquisition parameters corresponding to the video acquisition device are converted according to the preset parameter conversion matrix to obtain candidate video acquisition parameters and determine the second region information corresponding to the risk region. Based on the candidate video acquisition parameters and the second region information, the image acquisition parameters corresponding to the risk region are constructed.

[0090] Step S312: Using the image acquisition device deployed at the operation location, acquire the risk image corresponding to the risk area according to the image acquisition parameters.

[0091] Step S314: Input the risk image into the risk detection model for processing to obtain the risk level of the risk area.

[0092] Step S316: If the risk level is Level 1, execute the alert and warning tasks corresponding to the risk area according to the risk level.

[0093] Step S318: If the risk level is Level 2, execute the alert task corresponding to the risk area according to the risk level.

[0094] Step S320: If the risk level is level three, execute the power outage alarm task corresponding to the risk area according to the risk level.

[0095] Optionally, the image acquisition device deployed at the operating location acquires the risk image corresponding to the risk area according to the image acquisition parameters, including: acquiring an initial risk image corresponding to the risk area according to the image acquisition parameters; detecting whether the initial risk image meets the risk detection conditions; if yes, using the initial risk image as the risk image corresponding to the risk area; if no, extracting candidate video frames from the monitoring video, using the candidate video frames as the target video frames, and performing the step of locating the risk area in the target video frames until a risk image that meets the risk detection conditions is acquired.

[0096] Optionally, the training of the risk detection model includes: acquiring sample risk images and the sample risk levels corresponding to the sample risk images; constructing sample expansion prompts based on the sample risk images and the sample risk levels, and inputting the sample expansion prompts into a large language model; using the large language model to recall expanded risk images from an external database according to the sample expansion prompts, and generating expanded risk levels corresponding to the expanded risk images; training the initial risk detection model based on the expanded risk images and the expanded risk levels until the risk detection model that meets the training stopping condition is obtained.

[0097] The risk detection method provided in this embodiment aims to improve the accuracy of risk detection in underground coal mines. After acquiring monitoring video of the operation location in the mine using video acquisition equipment, risk detection can be completed based on the monitoring video through subsequent automated processing. In this process, target video frames can be extracted from the monitoring video. If a risk is initially identified at the operation location based on the target video frames, the risk area can be located in the target video frames, and the corresponding image acquisition parameters can be determined. Then, a high-precision image acquisition device can be used to acquire the risk image corresponding to the risk area according to the image acquisition parameters. After acquiring a high-precision risk image for the risk area, the risk image can be input into the risk detection model for processing to obtain the risk level of the risk area. This enables the model to quickly and accurately determine the risk level of the risk area, allowing for the subsequent execution of alarm tasks corresponding to the risk level. This risk detection architecture, combining image acquisition equipment with video acquisition equipment, effectively improves the accuracy of risk detection. Furthermore, by combining it with a machine learning model for risk prediction, the speed of risk detection can be improved, thereby ensuring the safety of underground coal mine operators.

[0098] Corresponding to the above method embodiments, this specification also provides embodiments of risk detection devices. Figure 4 A schematic diagram of a risk detection device according to one embodiment of this specification is shown.Figure 4 As shown, the device includes:

[0099] The acquisition module 402 is configured to acquire monitoring videos collected by the video acquisition device at the underground operation location of the coal mine, and extract target video frames from the monitoring videos.

[0100] The determination module 404 is configured to locate a risk area in the target video frame and determine the image acquisition parameters corresponding to the risk area if the operation position is identified as having a risk based on the target video frame.

[0101] The acquisition module 406 is configured to be an image acquisition device deployed at the operation location, and to acquire a risk image corresponding to the risk area according to the image acquisition parameters;

[0102] The processing module 408 is configured to input the risk image into the risk detection model for processing, obtain the risk level of the risk area, and execute the alarm task corresponding to the risk area according to the risk level.

[0103] In an optional embodiment, determining the image acquisition parameters corresponding to the risk area includes:

[0104] When the video acquisition device and the image acquisition device are located in the same position, the video acquisition parameters corresponding to the video acquisition device are obtained, and the first region information corresponding to the risk area is determined. The image acquisition parameters corresponding to the risk area are then constructed based on the video acquisition parameters and the first region information. When the video acquisition device and the image acquisition device are located in different positions, the video acquisition parameters corresponding to the video acquisition device are transformed according to a preset parameter transformation matrix to obtain candidate video acquisition parameters. The second region information corresponding to the risk area is then determined, and the image acquisition parameters corresponding to the risk area are then constructed based on the candidate video acquisition parameters and the second region information.

[0105] In an optional embodiment, extracting the target video frame from the surveillance video includes:

[0106] The monitoring video is processed by frame segmentation to obtain a video frame sequence. Video frames with a resolution greater than a resolution threshold are selected from the video frame sequence as target video frames. The step of locating a risk area in the target video frame when the operation location is identified as having a risk based on the target video frame includes: inputting the target video frame into a risk classification model for processing, and performing the step of locating the risk area in the target video frame when the processing result determines that the operation location has a risk.

[0107] In an optional embodiment, executing the alarm task corresponding to the risk area according to the risk level includes:

[0108] When the risk level is Level 1, the alert and alarm task corresponding to the risk area is executed according to the risk level; when the risk level is Level 2, the prompt and alarm task corresponding to the risk area is executed according to the risk level; when the risk level is Level 3, the power outage alarm task corresponding to the risk area is executed according to the risk level.

[0109] In an optional embodiment, before the step of inputting the risk image into a risk detection model for processing to obtain the risk level of the risk area is executed, the method further includes:

[0110] A reference image that has a time-aligned relationship with the risk image is identified; an image restoration feature corresponding to the risk image is constructed based on the reference image; the risk image is updated using the image restoration feature to obtain a target risk image; the target risk image is used as the risk image, and the risk image is input into a risk detection model for processing to obtain the risk level of the risk region.

[0111] In an optional embodiment, the image acquisition device deployed at the operating location acquires a risk image corresponding to the risk area according to the image acquisition parameters, including:

[0112] The image acquisition device deployed at the operation location acquires an initial risk image corresponding to the risk area according to the image acquisition parameters; it detects whether the initial risk image meets the risk detection conditions; if so, the initial risk image is used as the risk image corresponding to the risk area; if not, candidate video frames are extracted from the monitoring video, and the candidate video frames are used as the target video frames. The step of locating the risk area in the target video frames is performed until a risk image that meets the risk detection conditions is acquired.

[0113] In an optional embodiment, training the risk detection model includes:

[0114] Obtain sample risk images and their corresponding sample risk levels; construct sample expansion prompts based on the sample risk images and their risk levels, and input the sample expansion prompts into a large language model; use the large language model to recall expanded risk images from an external database according to the sample expansion prompts, and generate expanded risk levels corresponding to the expanded risk images; train an initial risk detection model based on the expanded risk images and their risk levels until a risk detection model that meets the training stopping condition is obtained.

[0115] The risk detection device provided in this embodiment aims to improve the accuracy of risk detection in underground coal mines. After acquiring monitoring video of the operation location in the mine using video acquisition equipment, it can perform risk detection based on the monitoring video through subsequent automated processing. During this process, target video frames can be extracted from the monitoring video. If a risk is initially identified at the operation location based on the target video frames, the risk area can be located within the target video frames, and the corresponding image acquisition parameters can be determined. Then, a high-precision image acquisition device can be used to acquire the risk image corresponding to the risk area according to the image acquisition parameters. After acquiring a high-precision risk image for the risk area, the risk image can be input into the risk detection model for processing to obtain the risk level of the risk area. This enables the model to quickly and accurately determine the risk level of the risk area, allowing for the execution of corresponding alarm tasks based on the risk level. This risk detection architecture, combining image acquisition equipment with video acquisition equipment, effectively improves risk detection accuracy. Furthermore, by combining it with a machine learning model for risk prediction, the speed of risk detection can be increased, thereby ensuring the safety of underground coal mine operators.

[0116] The above is an illustrative scheme of a risk detection device according to this embodiment. It should be noted that the technical solution of this risk detection device and the technical solution of the risk detection method described above belong to the same concept. For details not described in detail in the technical solution of the risk detection device, please refer to the description of the technical solution of the risk detection method described above.

[0117] Figure 5 A structural block diagram of a computing device 500 according to one embodiment of this specification is shown. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0118] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 540 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0119] In one embodiment of this specification, the above-described components of the computing device 500 and Figure 5 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 5 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0120] The computing device 500 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 500 can also be a mobile or stationary server.

[0121] The processor 520 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the risk detection method described above.

[0122] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the risk detection method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the risk detection method described above.

[0123] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the risk detection method described above.

[0124] The above is an illustrative embodiment of a computer-readable storage medium. It should be noted that the technical solution of this storage medium and the technical solution of the aforementioned risk detection method belong to the same concept. Details not described in detail in the technical solution of the storage medium can be found in the description of the technical solution of the aforementioned risk detection method.

[0125] An embodiment of this specification also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the risk detection method described above.

[0126] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the risk detection method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the risk detection method described above.

[0127] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0128] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0129] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0130] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0131] The preferred embodiments disclosed above are merely illustrative of this specification. Optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described in this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification.

Claims

1. A risk detection method, characterized in that, include: The video acquisition device collects monitoring video of the operation position in the coal mine, and performs frame segmentation processing on the monitoring video to obtain a video frame sequence. The video frame with a resolution greater than the resolution threshold is selected from the video frame sequence as the target video frame. The target video frame is input into a risk classification model for processing. If the processing result determines that there is a risk at the operation location, the risk area is located in the target video frame. When the video acquisition device and the image acquisition device are located in the same position, the video acquisition parameters corresponding to the video acquisition device are obtained, and the first area information corresponding to the risk area is determined. Based on the video acquisition parameters and the first area information, the image acquisition parameters corresponding to the risk area are constructed. When the video acquisition device and the image acquisition device are not located in the same position, the video acquisition parameters corresponding to the video acquisition device are converted according to a preset parameter conversion matrix to obtain candidate video acquisition parameters and determine the second region information corresponding to the risk region. Based on the candidate video acquisition parameters and the second region information, the image acquisition parameters corresponding to the risk region are constructed. The image acquisition device deployed at the operation location acquires the risk image corresponding to the risk area according to the image acquisition parameters. The risk image is input into the risk detection model for processing to obtain the risk level of the risk area, and the alarm task corresponding to the risk area is executed according to the risk level.

2. The risk detection method according to claim 1, characterized in that, The step of executing the alarm task corresponding to the risk area according to the risk level includes: If the risk level is Level 1, execute the alert and warning task corresponding to the risk area according to the risk level. If the risk level is level 2, execute the alert and warning task corresponding to the risk area according to the risk level. When the risk level is level three, the power outage alarm task corresponding to the risk area shall be executed according to the risk level.

3. The risk detection method according to claim 1, characterized in that, Before the step of inputting the risk image into the risk detection model for processing to obtain the risk level of the risk area is executed, the method further includes: Determine a reference image that has a time-aligned relationship with the risk image; Construct image restoration features corresponding to the risk image based on the reference image; The risk image is updated using the image restoration features to obtain the target risk image; The target risk image is used as the risk image, and the risk image is input into the risk detection model for processing to obtain the risk level of the risk area.

4. The risk detection method according to claim 1, characterized in that, The image acquisition device deployed at the operation location acquires risk images corresponding to the risk area according to the image acquisition parameters, including: The image acquisition device deployed at the operation location acquires an initial risk image corresponding to the risk area according to the image acquisition parameters. Detect whether the initial risk image meets the risk detection conditions; If so, the initial risk image shall be used as the risk image corresponding to the risk area; If not, extract candidate video frames from the surveillance video, use the candidate video frames as the target video frames, and perform the step of locating risk areas in the target video frames until a risk image that meets the risk detection conditions is acquired.

5. The risk detection method according to any one of claims 1 to 4, characterized in that, The training of the risk detection model includes: Obtain the sample risk image and the sample risk level corresponding to the sample risk image; Based on the sample risk image and the sample risk level, construct sample expansion prompt words, and input the sample expansion prompt words into the large language model; Using the large language model, expand risk images are retrieved from an external database according to the sample expansion prompt words, and the expansion risk level corresponding to the expansion risk image is generated; The initial risk detection model is trained based on the expansion risk image and the expansion risk level until the risk detection model that meets the training stopping condition is obtained.

6. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 5.

8. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 5.

Images