Safety early warning method and device based on video analysis, equipment and storage medium

Abstract

The invention discloses a safety early warning method and device based on video analysis, equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring multiple paths of monitoring videos of an industrial production site; for each path of monitoring video, selecting at least one model for identifying the monitoring video from the plurality of candidate models based on the configuration parameters corresponding to the monitoring video; wherein the configuration parameters are used for indicating identification tasks corresponding to the monitoring video, and different identification tasks are processed by adopting different models; identifying the monitoring video by adopting at least one model to obtain an identification result corresponding to the monitoring video; and generating safety early warning information of the industrial production site based on the identification results corresponding to the multiple paths of monitoring videos. According to the method, real-time early warning of risk factors is realized through analysis of multiple paths of monitoring videos, and the accuracy and timeliness of risk early warning in an industrial production field are improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a security warning method, device, equipment and storage medium based on video analysis. Background Art

[0002] With the development of the digital age, information management has emerged in industrial production, and a large number of digital, automated and intelligent ways have emerged in the industry to improve organizational operations, increase production efficiency and ensure production safety.

[0003] In current industrial production, safety inspections and supervision of production sites are carried out through a combination of manual inspections and observation of surveillance videos.

[0004] However, the relevant technology is unable to inspect and supervise all production sites simultaneously during safety inspections and supervisions, resulting in problems being missed and not discovered in a timely manner. Summary of the invention

[0005] The embodiment of the present application provides a security warning method, device, equipment and storage medium based on video analysis. The technical solution is as follows:

[0006] According to one aspect of an embodiment of the present application, a security early warning method based on video analysis is provided, the method comprising:

[0007] Obtain multi-channel surveillance videos of industrial production sites;

[0008] For each of the surveillance videos, based on the configuration parameters corresponding to the surveillance video, at least one model for identifying the surveillance video is selected from a plurality of candidate models; wherein the configuration parameters are used to indicate the identification task corresponding to the surveillance video, and different identification tasks are processed using different models;

[0009] Using the at least one model to identify the surveillance video, and obtaining an identification result corresponding to the surveillance video;

[0010] Based on the recognition results corresponding to the multiple surveillance videos, safety warning information of the industrial production site is generated.

[0011] According to one aspect of an embodiment of the present application, a security warning device based on video analysis is provided, the device comprising:

[0012] An acquisition module is used to acquire multi-channel surveillance videos of industrial production sites;

[0013] A configuration module, configured to select, for each of the surveillance videos, at least one model from a plurality of candidate models for identifying the surveillance video based on configuration parameters corresponding to the surveillance video; wherein the configuration parameters are used to indicate the identification task corresponding to the surveillance video, and different identification tasks are processed using different models;

[0014] A recognition module, configured to recognize the surveillance video using the at least one model to obtain a recognition result corresponding to the surveillance video;

[0015] The early warning module is used to generate safety early warning information of the industrial production site based on the recognition results corresponding to the multiple monitoring videos.

[0016] According to one aspect of an embodiment of the present application, a computer device is provided, the computer device comprising a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above method.

[0017] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the above method.

[0018] The technical solution provided by the embodiments of the present application includes at least the following beneficial effects:

[0019] After acquiring multiple surveillance videos of an industrial production site, for each surveillance video, based on the configuration parameters corresponding to the surveillance video, at least one model corresponding to the configuration parameters is selected, the surveillance video is identified using the model to obtain the corresponding identification results, and safety warning information for the industrial production site is generated based on the identification results. In this way, multiple surveillance videos can be analyzed, real-time warnings of hazardous factors can be achieved, and the accuracy and timeliness of hazard warnings in industrial production sites can be improved.

[0020] By selecting at least one model from candidate models according to the configuration information corresponding to multiple surveillance videos, the surveillance videos are identified and analyzed, thereby achieving security supervision of multiple surveillance videos at the same time, reducing labor costs and the problem of missing problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application;

[0022] Figure 2 is a flow chart of a security warning method based on video analysis provided by an embodiment of the present application;

[0023] Figure 3It is a schematic diagram of a process of generating security warning information provided by an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of a data node provided by an embodiment of the present application;

[0025] Figure 5 is a program flow chart of a security warning method based on video analysis provided by an embodiment of the present application;

[0026] Figure 6 It is a schematic diagram of a three-layer architecture of a security early warning method based on video analysis provided by an embodiment of the present application;

[0027] Figure 7 is a block diagram of a security warning device based on video analysis provided by an embodiment of the present application;

[0028] Figure 8 It is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0030] Edge devices refer to physical devices or sensors located at the edge of the computing network. They can be deployed on user devices, sensors, industrial equipment, smart phones, IoT devices, etc. They are connected to the cloud or central server through the local network to upload data to the cloud for analysis and storage. Edge devices are located outside the remote cloud computing resources, close to the data source and end users, and are used to collect, process and store data, and perform some local computing tasks to meet specific applications or needs.

[0031] Please refer to Figure 1 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment may include: a computer device 10.

[0032] In the method provided in the embodiment of the present application, the execution subject of each step may be a computer device 10, which is an electronic device with data calculation, processing and storage functions. The computer device 10 may be either a terminal device or a server.

[0033] Exemplarily, terminal devices include but are not limited to edge devices, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, game consoles, wearable devices, multimedia playback devices, augmented reality (AR) devices, virtual reality (VR) devices and other electronic devices.

[0034] Exemplarily, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as edge computing services, cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), as well as big data and artificial intelligence platforms, but is not limited to these.

[0035] like Figure 1 As shown, the computer device 10 can automatically identify the risk factors in the multi-channel monitoring video stream 11 by executing the method provided in the embodiment of the present application, and obtain the early warning result 12 corresponding to the multi-channel monitoring video stream 11. Among them, the early warning result 12 includes the identified risk factors and related prompts, and the risk factors are obtained by the computer device 10 through reasoning and analysis of the multi-channel monitoring video stream 11.

[0036] The embodiments of the present application can be applied to various scenarios, including but not limited to industrial production safety supervision, smart homes, public safety, etc., and the present application does not limit this.

[0037] Please refer to Figure 2 , which shows a flow chart of a security warning method based on video analysis provided by an embodiment of the present application. The method may include at least one of the following steps (210-240):

[0038] Step 210, obtaining multi-channel monitoring videos of the industrial production site.

[0039] Industrial production sites refer to the actual operation sites in the manufacturing and production process, usually located in factories, industrial facilities or manufacturing enterprises, used to manufacture products, assemble parts, process raw materials or perform other industrial production tasks. Industrial production sites include various types of production equipment, including but not limited to the following equipment: machine tools, production lines, assembly lines, conveyor belts, robotic arms, etc. The above production equipment is operated by relevant staff.

[0040] Multi-channel surveillance video refers to the surveillance video recorded by the surveillance cameras in the industrial production site. Multi-channel refers to the surveillance video from multiple different cameras. The surveillance cameras in the industrial production site are connected to the network through wired or wireless fidelity (Wireless Fidelity, referred to as WI-FI), and the surveillance video stream is transmitted to the monitoring device through the network. The monitoring device refers to the device used to view, record and manage the surveillance video stream transmitted by the surveillance camera, such as Figure 1 The computer device 10 in.

[0041] In some embodiments, the surveillance camera in the industrial production site has certain image processing, encoding, and storage capabilities, and can store the surveillance video in the form of a video file in the surveillance camera.

[0042] In some embodiments, a surveillance video from a surveillance camera corresponds to one surveillance video among multiple surveillance videos.

[0043] In some embodiments, the surveillance video of the surveillance camera is remotely called through the Internet Protocol Address (IP address) of the surveillance camera. It should be noted that an IP address is uniquely assigned to a surveillance camera.

[0044] For example, assume that there are three surveillance cameras: surveillance camera A, surveillance camera B and surveillance camera C, and their corresponding IP addresses are: address a, address b and address c, respectively. Surveillance camera A and surveillance camera C are selected for remote access, that is, the corresponding surveillance videos are called according to their IP addresses, that is, two surveillance videos are obtained according to address a and address c.

[0045] Through the above method, multiple surveillance videos that need to be monitored for security are obtained, providing surveillance video data for subsequent video analysis.

[0046] Step 220, for each surveillance video, based on the configuration parameters corresponding to the surveillance video, select at least one model for identifying the surveillance video from multiple candidate models; wherein the configuration parameters are used to indicate the identification task corresponding to the surveillance video, and different identification tasks are processed using different models.

[0047] Configuration parameters are parameters set according to application scenarios, and may include at least one of the following parameters: application scenario, IP address of the corresponding surveillance camera, third-party monitoring equipment, etc. The specific parameters are set by relevant technical personnel and are not limited in this application. The application scenarios of the method of this application at industrial production sites include but are not limited to the following scenarios: detecting people who are not wearing safety helmets, detecting people who are not wearing reflective clothing, detecting flames, detecting smoke, range intrusion warning, crossing the line and boundary warning, detecting fallen people, etc. The specific application scenarios are not limited in this application.

[0048] The recognition task refers to the task of automatically identifying or locating a specified object, subject or area in a surveillance video, including but not limited to the following tasks: identifying humans, identifying helmets, identifying human actions, identifying human postures, identifying dangerous objects, identifying fire ranges, etc., which are not limited in this application.

[0049] In some embodiments, the recognition task corresponding to the application scenario is set by relevant technical personnel, and the corresponding model parameters are set in the corresponding configuration parameters.

[0050] In some embodiments, an application scenario corresponds to at least one recognition task.

[0051] In some embodiments, a surveillance video may be configured with at least one application scenario, or may not be configured with an application scenario. Exemplarily, when a surveillance video is configured with at least one application scenario, corresponding models are configured for different application scenarios according to configuration parameters, and the surveillance video is pushed to the corresponding model. Exemplarily, when a surveillance video is not configured with an application scenario, the surveillance video is not identified and analyzed.

[0052] Optionally, the configuration parameters can be set by the user, or the default configuration parameters can be set by relevant technical personnel. Optionally, in the case where the configuration parameters are set by the user, the user can change the configuration parameters at any time, and the selected model will also change accordingly. Exemplarily, the user chooses to apply the method of the present application to the scene of detecting people who are not wearing helmets. For this scene, it is necessary to complete the task of identifying people and helmets, then the configuration parameters include model parameters related to identifying people and helmets.

[0053] In some embodiments, based on the configuration parameters corresponding to the surveillance video, the recognition task corresponding to the surveillance video is determined; according to the recognition tasks corresponding to multiple candidate models, at least one model for processing the recognition task corresponding to the surveillance video is selected from the multiple candidate models as at least one model for recognizing the surveillance video.

[0054] In some embodiments, the candidate model is an artificial intelligence (AI) model that can recognize or locate objects, subjects or areas from a video.

[0055] By completing the corresponding recognition task through the above method, the specified object can be identified and located from the surveillance video to prepare for subsequent analysis.

[0056] Step 230: Use at least one model to identify the surveillance video to obtain an identification result corresponding to the surveillance video.

[0057] In some embodiments, key frames of the surveillance video are obtained and input into at least one of the models to obtain recognition results corresponding to the surveillance video.

[0058] The recognition result is information related to the recognition task, including but not limited to at least one of the following information: the position, category, confidence information, degree of occlusion, posture, speed, etc. of the recognition object.

[0059] In some embodiments, when at least one model is one model, one model is used to identify the surveillance video to obtain the recognition result corresponding to the surveillance video; or, when at least one model is multiple models, based on the association relationship between the multiple models, multiple models are used to identify the surveillance video to obtain the recognition result corresponding to the surveillance video.

[0060] In some embodiments, if there are at least two surveillance videos that use the same model, only one model is started, that is, the two surveillance videos use the same model for calculation.

[0061] The above method is helpful to save computing and memory resources.

[0062] In some embodiments, the association relationship includes a series relationship and a parallel relationship; for two models with a series relationship, the input data of one model depends on the output data of the other model; for two models with a parallel relationship, the input data of the two models are independent of each other.

[0063] In some embodiments, the models corresponding to the configuration parameters can be combined in multiple series and parallel relationships to complete the recognition task.

[0064] Exemplarily, assuming that the models corresponding to the recognition task are model A and model B, which have a series relationship, the output of model A must be further processed before the input of model B can be obtained, that is, the input of model B is the output after the above processing.

[0065] Exemplarily, it is assumed that the models corresponding to the recognition task are model A and model B, which have a parallel relationship, and the input of model A and model B is the same surveillance video.

[0066] Exemplarily, assuming that the models corresponding to the recognition task are model A, model B and model C, model A has a series relationship with model B and a parallel relationship with model C. The outputs of model A and model B must be further processed before the input of model C can be obtained, that is, the input of model C is the output after the above processing.

[0067] Through the above method, by combining multiple models, more complex recognition tasks can be completed and can be applied to more scenarios.

[0068] For example, it is assumed that the models corresponding to the recognition task are model A and model B, which have a series relationship. Therefore, the surveillance video is input into model A to obtain the output result, and the processed output result is input into model B to obtain the output of model B. At this time, the recognition task is completed.

[0069] Exemplarily, assume that the models corresponding to the recognition task are model A and model B, which have a parallel relationship. The input of model A and model B is the same surveillance video. The surveillance video is input into model A and model B respectively, and their output results are obtained respectively. At this time, the recognition task is completed.

[0070] Exemplarily, it is assumed that the models corresponding to the recognition task are model A, model B and model C. Model A has a series relationship with model B and a parallel relationship with model C. Therefore, the surveillance video is input into model A and model B respectively, and their output results are obtained respectively. After model A and model B are integrated according to the application scenario, the processed output results are input into model C to obtain the output results of model C. At this time, the recognition task is completed.

[0071] Through the above method, surveillance videos are identified and detected based on the model, which improves the speed and accuracy of identification and is conducive to timely warning of dangerous behaviors.

[0072] Step 240: Generate safety warning information for the industrial production site based on the recognition results corresponding to the multiple surveillance videos.

[0073] In some embodiments, a business algorithm is used to analyze the recognition results corresponding to multiple surveillance videos to obtain analysis results; wherein the business algorithm is used to define the method for analyzing the recognition results; based on the analysis results, safety warning information for the industrial production site is generated.

[0074] The business algorithm is the logic for analyzing, judging, and processing the recognition results based on the application scenario. For example, based on the recognition results of the person and the helmet, it determines whether the person appearing in the surveillance video is wearing the helmet correctly.

[0075] For example, Figure 3 , which shows a schematic diagram of the process of generating safety warning information provided by an embodiment of the present application. Assuming that the application scenario is to detect people who are not wearing helmets, sub-graph (a) is a key frame extracted from the surveillance video. After identifying the key frame, the recognition result of the person and the helmet is obtained. The business algorithm is used to analyze the recognition result to determine whether the person appearing in the key frame corresponding to the surveillance video is wearing a helmet. If there is a person who is not wearing a helmet in the key frame, a safety warning information is obtained, i.e., sub-graph (b), and the person 31 who is not wearing a helmet is identified.

[0076] In some embodiments, based on the association relationship and identification tasks between the models, business algorithms also exist between the models.

[0077] Exemplarily, model A is a model for identifying people, model B is a model for identifying mobile phones, and model C is a model for identifying human postures. Model A and model B have a parallel relationship, and also have a series relationship with model C. After obtaining the output results of model A and model B, the business algorithm is used to determine whether there is a person in the surveillance video and whether there is a mobile phone in the surveillance video. When both of the above judgments are true, the above output results are integrated and input into model C.

[0078] Through the above method, by using business algorithms to analyze the recognition results, early warning results can be quickly obtained to achieve timely warnings and improve production safety.

[0079] In some embodiments, the safety warning information includes at least one of a warning picture and a warning video, and image content with safety hazards in the warning picture and the warning video is marked and displayed.

[0080] There are many ways to mark and display image content that poses a safety hazard in warning pictures and warning videos, which may include but are not limited to any of the following methods: prompt text, symbols, wireframes, etc., which are not limited in this application.

[0081] For example, Figure 3 As shown in sub-figure (b), the person 31 who is not wearing a helmet is displayed using a wireframe 32 and a symbol “!” 33.

[0082] Through the above method, the safety warning information is displayed intuitively and eye-catchingly.

[0083] Optionally, when generating the safety warning information, if there is a dangerous factor in the monitoring video, a first message is sent. The first message can be a warning sound, or a warning text, light, etc.

[0084] Optionally, the first information may be sent to other monitoring devices.

[0085] In some embodiments, the method also includes the steps of: using data nodes as the smallest unit, storing relevant information corresponding to the monitoring video; wherein the relevant information includes video frame data, scene data and business algorithm data; the video frame data is used to indicate at least one video frame in the monitoring video, the scene data is used to indicate at least one model for identifying at least one video frame, and the business algorithm data is used to indicate the business algorithm used to analyze the recognition results.

[0086] A video frame refers to a key frame extracted from a surveillance video, which contains information related to the surveillance video.

[0087] In some embodiments, the video frame data includes corresponding frame number, frame data, and frame size. The frame number indicates the position of the video frame in the surveillance video, the frame size indicates the storage space occupied by the video frame, and the frame data is the video frame itself.

[0088] In some embodiments, the scene data includes a model serial number, a model path, a model type, and combined model information of at least one model for identifying at least one video frame.

[0089] In some embodiments, the business algorithm data includes business algorithms, algorithm paths, configuration information, parameters, and business algorithm results.

[0090] For example, Figure 4 As shown, it shows a schematic diagram of a data node provided by an embodiment of the present application. A data node stores three types of data: video frame data, scene data and business algorithm data, and stores their corresponding information respectively.

[0091] Through the above method, relevant information is encapsulated into a data node, which is beneficial to data access during the identification and analysis process, improves analysis efficiency, and reduces the coupling between steps.

[0092] To summarize, the technical solution provided in the embodiment of the present application, after acquiring multiple surveillance videos of an industrial production site, for each surveillance video, based on the configuration parameters corresponding to the surveillance video, selects at least one model corresponding to the configuration parameters, uses the model to identify the surveillance video, and obtains the corresponding identification results. Based on the identification results, safety warning information of the industrial production site is generated, thereby analyzing multiple surveillance videos, realizing real-time warning of dangerous factors, and improving the accuracy and timeliness of danger warnings in industrial production sites.

[0093] By selecting at least one model from candidate models according to the configuration information corresponding to multiple surveillance videos, the surveillance videos are identified and analyzed, thereby achieving security supervision of multiple surveillance videos at the same time, reducing labor costs and the problem of missing problems.

[0094] Please refer to Figure 5 , which shows a program flow chart of a security warning method based on video analysis provided by an embodiment of the present application.

[0095] After extracting the corresponding key frames, i.e., video frames, from multiple surveillance videos, data nodes are generated according to the information and configuration parameters of the video frames of each surveillance video; the model used to analyze each surveillance video is parsed from the data node, and the model is selected from the candidate model list to determine whether the model needs to be loaded. If the required model has been loaded and can still meet the calculation of the surveillance video, the model does not need to be loaded, otherwise the model is loaded; the video frames of the surveillance video are recognized and analyzed using the selected model to obtain the recognition result; if the recognition result needs to be processed before entering the next step in this application scenario, the corresponding algorithm is used to process the recognition result; if the selected model includes more than one model, the model needs to be combined, and then the model is selected and the above steps are repeated; after obtaining the recognition results of all selected models, the recognition results of all selected models are integrated and calculated using the business algorithm to obtain the final security warning result.

[0096] Please refer to Figure 6 , which shows a schematic diagram of a three-layer architecture of a security warning method based on video analysis provided by an embodiment of the present application.

[0097] The security warning method based on video analysis can be divided into three layers in the processing flow: a user layer, a model reasoning layer, and a business post-processing layer. The model reasoning layer can include steps 210, 220, and 230, and the business post-processing layer can include step 240.

[0098] At the user level, users can set configuration parameters to determine the application scenarios of the security warning method based on video analysis; users can view the security warning results and call the analysis results of the security warning method based on video analysis.

[0099] At the model inference layer, multiple surveillance videos are imported according to the configuration parameters set by the user, and each surveillance video is decoded to obtain their corresponding video frames. The video frames are distributed to the corresponding models for recognition, and the recognition results corresponding to the video frames are obtained. It is determined whether the corresponding video frames need to be warned, and the warning pictures and short videos of the video frames that need to be warned are obtained.

[0100] In the business post-processing layer, the recognition results obtained in the model reasoning layer are reasoned and analyzed to obtain and encapsulate the security warning results.

[0101] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0102] Please refer to Figure 7, which shows a block diagram of a security warning device based on video analysis provided by an embodiment of the present application. The device has the function of implementing the above method example, and the function can be implemented by hardware or by hardware executing corresponding software. The device can be the computer device introduced above, or it can be set in a computer device. Figure 7 As shown, the device 700 may include an acquisition module 710 , a configuration module 720 , an identification module 730 and an early warning module 740 .

[0103] The acquisition module 710 is used to acquire multi-channel monitoring videos of the industrial production site.

[0104] Configuration module 720 is used to select at least one model for identifying the surveillance video from multiple candidate models for each surveillance video based on the configuration parameters corresponding to the surveillance video; wherein the configuration parameters are used to indicate the recognition task corresponding to the surveillance video, and different recognition tasks are processed using different models.

[0105] The recognition module 730 is used to use the at least one model to recognize the surveillance video and obtain a recognition result corresponding to the surveillance video.

[0106] The early warning module 740 is used to generate safety early warning information of the industrial production site based on the recognition results corresponding to the multiple surveillance videos.

[0107] In some embodiments, the configuration module 720 is used to determine the recognition task corresponding to the surveillance video based on the configuration parameters corresponding to the surveillance video; according to the recognition tasks corresponding to the multiple candidate models, at least one model for processing the recognition task corresponding to the surveillance video is selected from the multiple candidate models as at least one model for recognizing the surveillance video.

[0108] In some embodiments, the recognition module 730 is used to, when the at least one model is one model, use the one model to recognize the surveillance video to obtain a recognition result corresponding to the surveillance video; or, when the at least one model is multiple models, based on the association relationship between the multiple models, use the multiple models to recognize the surveillance video to obtain a recognition result corresponding to the surveillance video.

[0109] In some embodiments, the association relationship includes a series relationship and a parallel relationship; in the two models having the series relationship, the input data of one model depends on the output data of the other model; in the two models having the parallel relationship, the input data of the two models are independent of each other.

[0110] In some embodiments, the early warning module 740 is used to analyze the recognition results corresponding to the multiple surveillance videos respectively using a business algorithm to obtain an analysis result; wherein the business algorithm is used to define a method for analyzing the recognition results; based on the analysis results, generate safety early warning information for the industrial production site.

[0111] In some embodiments, the device 700 further includes a storage module ( Figure 7 (not shown in the figure), which is used to store the relevant information corresponding to the monitoring video with the data node as the smallest unit; wherein the relevant information includes video frame data, scene data and business algorithm data; the video frame data is used to indicate at least one video frame in the monitoring video, the scene data is used to indicate at least one model for identifying the at least one video frame, and the business algorithm data is used to indicate the business algorithm used to analyze the recognition result.

[0112] In some embodiments, the safety warning information includes at least one of a warning picture and a warning video, and image contents with safety hazards in the warning picture and the warning video are marked and displayed.

[0113] To summarize, the technical solution provided in the embodiment of the present application, after acquiring multiple surveillance videos of an industrial production site, for each surveillance video, based on the configuration parameters corresponding to the surveillance video, selects at least one model corresponding to the configuration parameters, uses the model to identify the surveillance video, and obtains the corresponding identification results. Based on the identification results, safety warning information of the industrial production site is generated, thereby analyzing multiple surveillance videos, realizing real-time warning of dangerous factors, and improving the accuracy and timeliness of danger warnings in industrial production sites.

[0114] By selecting at least one model from candidate models according to the configuration information corresponding to multiple surveillance videos, the surveillance videos are identified and analyzed, thereby achieving security supervision of multiple surveillance videos at the same time, reducing labor costs and the problem of missing problems.

[0115] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0116] Please refer to Figure 8, which shows a block diagram of a computer device 800 provided in one embodiment of the present application. The computer device 800 may be Figure 1 The computer device 10 in the implementation environment shown is used to implement the security early warning method based on video analysis provided in the above embodiment. Specifically:

[0117] Typically, the computer device 800 includes a processor 810 and a memory 820 .

[0118] The processor 810 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 810 may be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 810 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 810 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 810 may also include an AI processor, which is used to process computing operations related to machine learning.

[0119] The memory 820 may include one or more computer-readable storage media, which may be non-transitory. The memory 820 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 820 is used to store a computer program, which is configured to be executed by one or more processors to implement the above-mentioned security warning method based on video analysis.

[0120] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the computer device 800, and the computer device 800 may include more or less components than those shown in the figure, or combine some components, or adopt a different arrangement of components.

[0121] In an exemplary embodiment, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned security warning method based on video analysis is implemented. Optionally, the computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid state drive (SSD) or an optical disk, etc. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM).

[0122] In an exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above-mentioned security early warning method based on video analysis.

[0123] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0124] The term “greater than or equal to” mentioned herein may mean greater than or equal to, or greater than, and the term “less than or equal to” may mean less than or equal to, or less than.

[0125] In addition, the step numbers described in this document only illustrate a possible execution order between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to that shown in the figure. The embodiments of the present application are not limited to this.

[0126] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A safety warning method based on video analysis, characterized in that: The method comprises: Obtain multi-channel surveillance videos of industrial production sites; For each of the surveillance videos, based on the configuration parameters corresponding to the surveillance video, at least one model for identifying the surveillance video is selected from a plurality of candidate models; wherein the configuration parameters are used to indicate the identification task corresponding to the surveillance video, and different identification tasks are processed using different models; Using the at least one model to identify the surveillance video, and obtaining an identification result corresponding to the surveillance video; Based on the recognition results corresponding to the multiple surveillance videos, safety warning information of the industrial production site is generated.

2. The method according to claim 1, characterized in that The selecting, based on the configuration parameters corresponding to the surveillance video, at least one model for identifying the surveillance video from a plurality of candidate models comprises: Determining a recognition task corresponding to the surveillance video based on configuration parameters corresponding to the surveillance video; According to the recognition tasks respectively corresponding to the multiple candidate models, at least one model for processing the recognition task corresponding to the surveillance video is selected from the multiple candidate models as at least one model for recognizing the surveillance video.

3. The method according to claim 1, characterized in that The adopting the at least one model to identify the surveillance video to obtain an identification result corresponding to the surveillance video includes: In the case where the at least one model is one model, using the one model to identify the surveillance video to obtain an identification result corresponding to the surveillance video; or, In the case where the at least one model is a plurality of models, based on the association relationship between the plurality of models, the plurality of models are used to identify the surveillance video to obtain an identification result corresponding to the surveillance video.

4. The method according to claim 3, characterized in that The association relationship includes a series relationship and a parallel relationship; Two models having the series relationship, wherein the input data of one model depends on the output data of the other model; The two models have the parallel relationship, and the input data of the two models are independent of each other.

5. The method according to claim 1, characterized in that The generating of safety warning information of the industrial production site based on the recognition results respectively corresponding to the multiple surveillance videos includes: Using a business algorithm to analyze the recognition results corresponding to the multiple surveillance videos to obtain analysis results; wherein the business algorithm is used to define a method for analyzing the recognition results; Based on the analysis results, safety warning information of the industrial production site is generated.

6. The method according to claim 1, characterized in that The method further comprises: The relevant information corresponding to the monitoring video is stored with the data node as the smallest unit; wherein the relevant information includes video frame data, scene data and business algorithm data; the video frame data is used to indicate at least one video frame in the monitoring video, the scene data is used to indicate at least one model for identifying the at least one video frame, and the business algorithm data is used to indicate the business algorithm used to analyze the identification result.

7. The method according to claim 1, characterized in that The safety warning information includes at least one of a warning picture and a warning video, and image contents with potential safety hazards in the warning picture and the warning video are marked and displayed.

8. A safety warning device based on video analysis, characterized in that: The device comprises: An acquisition module is used to acquire multi-channel surveillance videos of industrial production sites; A configuration module, configured to select, for each of the surveillance videos, at least one model from a plurality of candidate models for identifying the surveillance video based on configuration parameters corresponding to the surveillance video; wherein the configuration parameters are used to indicate the identification task corresponding to the surveillance video, and different identification tasks are processed using different models; A recognition module, configured to recognize the surveillance video using the at least one model to obtain a recognition result corresponding to the surveillance video; The early warning module is used to generate safety early warning information of the industrial production site based on the recognition results corresponding to the multiple monitoring videos.

9. A computer device, characterized in that: The computer device comprises a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to be executed by a processor to implement the method according to any one of claims 1 to 7.

Images