Safety control method, system and device based on deep learning, and electronic equipment

Abstract

According to the security control method, system and device based on deep learning, and the electronic equipment provided by the invention, the to-be-recognized image corresponding to the target area is acquired, and the to-be-recognized image is input into the trained image recognition model for image recognition processing to obtain the image recognition result; wherein the to-be-recognized image at least comprises a first monitoring target and a second monitoring target; and generating corresponding early warning information according to the image recognition result and the to-be-recognized image, and transmitting the early warning information to a preset terminal. An image acquired on site is analyzed through a trained image recognition model, early warning information is generated according to an analysis result (an image recognition result), and the early warning information is transmitted to portable equipment or an intelligent management platform (a preset terminal), so that intelligent recognition and automatic warning of a monitoring video of a working site can be realized, and the working efficiency is improved. Safety management and control can be automatically performed on a plurality of working points, on-site management and control can be performed without a safety supervisor, and the efficiency and the accuracy of safety management and control can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a security management and control method and system, device, and electronic device based on deep learning. Background Art

[0002] With the rapid development of my country's economy, in order to improve the level of production safety in high-risk industries such as construction, chemical, and mining, the state and enterprises have invested a lot of manpower and material resources in safety management, but traditional safety management and control methods still have certain limitations.

[0003] At present, traditional personnel operation safety management and control technologies mainly include the following: Video monitoring: By installing cameras at the work site, the behavior of workers is monitored in real time. However, this method has blind spots in monitoring and cannot provide early warning of potential dangers. Sensor monitoring: Use various sensors to monitor dangerous factors in the working environment, such as gas concentration, temperature, etc. However, the sensor monitoring range is limited and cannot effectively monitor personnel behavior. Manual inspection: Arrange special personnel to conduct on-site inspections to discover safety hazards and deal with them in a timely manner. However, manual inspections have problems such as high labor intensity, low efficiency, and susceptibility to subjective factors.

[0004] Therefore, how to solve the problems of too many security control work points, insufficient security supervision personnel, and low efficiency of manual video monitoring is an urgent problem that needs to be solved. Summary of the invention

[0005] The present invention provides a deep learning-based security control method and system, device, and electronic device, which are mainly intended to solve the problems of too many security control work points, insufficient security supervisors, and low efficiency of manual video monitoring.

[0006] According to a first aspect of the present disclosure, a security management and control method based on deep learning is provided, which includes:

[0007] Obtain an image to be identified corresponding to the target area, and input the image to be identified into a trained image recognition model for image recognition processing to obtain an image recognition result; wherein the image to be identified includes at least a first monitoring target and a second monitoring target;

[0008] Generate corresponding warning information according to the image recognition result and the image to be recognized, and transmit the warning information to a preset terminal.

[0009] Optionally, the generating corresponding warning information according to the image recognition result and the image to be recognized includes:

[0010] When it is determined according to the image recognition result that the first monitoring target does not meet the first preset condition, generating a first warning information;

[0011] When it is determined according to the image recognition result that the second monitoring target does not meet the second preset condition, a second warning information is generated and an alarm is issued through a preset alarm device.

[0012] Optionally, before acquiring the image to be recognized corresponding to the target area, the method further includes:

[0013] Acquire training image data; wherein the training image data includes training images to be identified and label data;

[0014] Inputting the training image to be recognized into the image recognition model for image recognition processing to obtain a training image recognition result;

[0015] According to the training image recognition result and the label data, the image recognition model is optimized by a preset loss function to obtain the trained image recognition model.

[0016] Acquire first target information of the first monitoring target and second target information of the second monitoring target, and configure the first target information and the second target information in the trained image recognition model.

[0017] Optionally, the step of inputting the image to be recognized into a trained image recognition model for image recognition processing to obtain an image recognition result includes:

[0018] Based on the first target information and the second target information, the image to be recognized is subjected to image recognition processing by the trained image recognition model to obtain the image recognition result.

[0019] Optionally, after transmitting the warning information to a preset terminal, the method further includes:

[0020] Performing visualization on the warning information;

[0021] In response to the triggered communication instruction, communication processing is performed with the first monitoring target through the preset terminal based on the early warning information.

[0022] According to a second aspect of the present disclosure, a security management and control system based on deep learning is provided, comprising: an acquisition device, an analysis device, and a preset terminal.

[0023] The acquisition device is used to acquire the image to be identified corresponding to the target area, and send the image to be identified to the analysis device;

[0024] The analysis device is used to input the image to be identified into a trained image recognition model to perform image recognition processing to obtain an image recognition result; wherein the image to be identified includes at least a first monitoring target and a second monitoring target;

[0025] The analysis device is also used to generate corresponding warning information according to the image recognition result and the image to be recognized, and transmit the warning information to the preset terminal.

[0026] Optionally, the preset terminal includes: a first terminal, a second terminal and a third terminal,

[0027] The first terminal and the second terminal are used to obtain first target information of the first monitoring target and second target information of the second monitoring target;

[0028] The first terminal and the second terminal are further used to visualize the warning information;

[0029] The first terminal and the second terminal are also used to, in response to the triggered communication instruction, communicate with the first monitoring target through the preset terminal based on the warning information;

[0030] The third terminal is used to visualize the warning information.

[0031] According to a third aspect of the present disclosure, a security management and control device based on deep learning is provided, comprising:

[0032] An acquisition unit, used for acquiring an image to be identified corresponding to a target area;

[0033] A processing unit, used for inputting the image to be identified into a trained image recognition model for image recognition processing to obtain an image recognition result; wherein the image to be identified includes at least a first monitoring target and a second monitoring target;

[0034] A generating unit, configured to generate corresponding warning information according to the image recognition result and the image to be recognized;

[0035] The transmission unit is used to transmit the warning information to a preset terminal.

[0036] Optionally, the generating unit includes:

[0037] A generating module, configured to generate first warning information when it is determined that the first monitoring target does not meet a first preset condition according to the image recognition result;

[0038] The generating module is further configured to generate second warning information when it is determined according to the image recognition result that the second monitoring target does not meet the second preset condition;

[0039] The alarm module is used to issue an alarm through a preset alarm device.

[0040] Optionally, the acquisition unit is further used to acquire training image data; wherein the training image data includes training images to be identified and label data;

[0041] The processing unit is further used to input the training image to be recognized into the image recognition model for image recognition processing to obtain a training image recognition result;

[0042] The device also includes:

[0043] The optimization unit is used to optimize the image recognition model through a preset loss function according to the training image recognition result and the label data to obtain the trained image recognition model.

[0044] The acquisition unit is further used to acquire first target information of the first monitoring target and second target information of the second monitoring target;

[0045] A configuration unit is used to configure the first target information and the second target information in the trained image recognition model.

[0046] Optionally, the processing unit is further used to perform image recognition processing on the image to be recognized through the trained image recognition model based on the first target information and the second target information to obtain the image recognition result.

[0047] Optionally, the device further comprises:

[0048] A display unit, used for visualizing the warning information;

[0049] A communication unit is used to respond to the triggered communication instruction and communicate with the first monitoring target through the preset terminal based on the early warning information.

[0050] According to a fourth aspect of the present disclosure, there is provided an electronic device, including:

[0051] at least one processor; and

[0052] a memory communicatively connected to the at least one processor; wherein,

[0053] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0054] According to a fifth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0055] According to a sixth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0056] The security control method and system, device, and electronic device based on deep learning provided by the present disclosure obtain an image to be identified corresponding to a target area, and input the image to be identified into a trained image recognition model for image recognition processing to obtain an image recognition result; wherein the image to be identified includes at least a first monitoring target and a second monitoring target; corresponding warning information is generated according to the image recognition result and the image to be identified, and the warning information is transmitted to a preset terminal. Compared with the related art, the embodiment of the present disclosure analyzes the image obtained on site through a trained image recognition model, generates warning information according to the analysis result (image recognition result), and transmits the warning information to a portable device or an intelligent management platform (preset terminal), which can realize intelligent recognition and automatic alarm of the monitoring video of the work site, can automatically perform safety control on multiple work points, and does not require on-site control by safety supervisors, which can improve the efficiency and accuracy of safety control.

[0057] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0059] Figure 1 A flowchart of a deep learning-based security management and control method provided in an embodiment of the present disclosure;

[0060] Figure 2 A schematic diagram of a generation process of early warning information provided by an embodiment of the present disclosure;

[0061] Figure 3 A schematic diagram of the structure of a deep learning-based security management and control system provided in an embodiment of the present disclosure;

[0062] Figure 4 A schematic diagram of an edge AI analysis device provided by an embodiment of the present disclosure;

[0063] Figure 5 A schematic diagram of a smart safety supervision platform provided by an embodiment of the present disclosure;

[0064] Figure 6 A schematic diagram of a networking mode provided by an embodiment of the present disclosure;

[0065] Figure 7 A schematic diagram of the structure of a deep learning-based security management and control device provided in an embodiment of the present disclosure;

[0066] Figure 8 A schematic diagram of the structure of another deep learning-based security management and control device provided in an embodiment of the present disclosure;

[0067] Fig. 9 A schematic block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0068] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0069] The following describes the deep learning-based security management method and system, device, and electronic device of the embodiments of the present disclosure with reference to the accompanying drawings.

[0070] Figure 1 A flowchart of a deep learning-based security management and control method provided in an embodiment of the present disclosure.

[0071] like Figure 1 As shown, the method comprises the following steps:

[0072] Step 101, obtain an image to be identified corresponding to the target area, and input the image to be identified into a trained image recognition model for image recognition processing to obtain an image recognition result; wherein the image to be identified includes at least a first monitoring target and a second monitoring target.

[0073] In an embodiment of the present disclosure, the target area is an area that requires security control, such as the working areas of various factories, etc. The image to be identified is a monitoring image of the target area. The types of the image to be identified include but are not limited to videos, pictures, etc. The first monitoring target is matters related to the staff in the target area, and the second monitoring target is various equipment in the target area, such as pipelines, machines, etc. Specifically, the embodiment of the present disclosure does not limit the target area, the image to be identified, the first monitoring target and the second monitoring target.

[0074] The trained image recognition model is a pre-trained model. The types of the trained image recognition model include but are not limited to: image recognition based on deep neural network technology, etc. The trained image recognition model can be based on the video images of on-site personnel operations and can simulate the working mode of the biological vision system to effectively extract the characteristics of personnel violations from the images, and accurately identify and timely warn dozens of violations such as not wearing a safety helmet.

[0075] The image recognition result is the on-site conditions of the target area identified based on the image to be identified, such as leakage, fire hazard, personnel violation and other risk hazards.

[0076] Among them, the items that the trained image recognition model can identify (i.e., specific image recognition results) include but are not limited to: 1. Failure to place identification signs in a standardized manner: for different on-site work contents, identify whether identification signs are placed in the specified position before the work, and whether the content of the identification signs meets the work requirements. If monitoring finds that the identification signs are not placed as required, an early warning will be issued; 2. Not wearing a safety helmet: for on-site construction work surfaces, identify workers who are not wearing safety helmets for timely monitoring and early warning, and support automatic identification of workers on the same work surface who are not wearing safety helmets; 3. Failure to dress in a standardized manner: for on-site construction work surfaces, identify workers who are not dressed in accordance with the requirements, such as whether the clothes and cuffs are buttoned, whether they wear scarves and long clothes, and automatically Automatically issue an early warning to remind managers and staff; 4. No safety belt at high altitude: For high-altitude work, when the construction workers are not wearing safety belts and safety ropes, an early warning will be automatically issued to remind managers and staff; 5. No ladder: For high-altitude work, when the workers are working on the insulating ladder, there is no one to help the ladder, and an early warning will be automatically issued to remind managers and staff; 6. No insulating gloves: Determine whether the on-site workers are wearing the required insulating gloves, and automatically issue an early warning to remind managers and staff; 7. No insulating shoes: Determine whether the on-site workers are wearing the required insulating shoes, and automatically issue an early warning to remind managers and staff; 8. Prohibited area intrusion: For important areas demarcated by electronic fences, set a prohibited intrusion time and conduct positioning monitoring. When a person intrudes or invades, an early warning is automatically issued to remind managers and staff; 9. Overcrowding in the area: Automatically associate the work ticket to obtain the number of staff in each construction area, and monitor in real time. When it is found that the actual number of staff in the area exceeds the specified number, an early warning is automatically issued to remind managers and staff; 10. Face recognition: Enter face information in the system in advance. When it is found that the on-site staff is a stranger, an early warning is automatically issued to remind managers and staff. The system supports liveness detection to prevent staff from impersonating others; 11. Leakage: Combined with sensor equipment and video surveillance Control equipment, timely detect leakage of important equipment and issue warnings, and notify relevant personnel to deal with it in time; 12. Fireworks detection: timely monitor and warn of smoke and flames at the work site, and notify relevant personnel to deal with it in time; 13. Smoking detection: real-time monitoring of whether there are cigarettes in the mouths and hands of staff. When smoking behavior is found, the alarm will be immediately reported to the relevant personnel to handle it, so as to assist management personnel in controlling smoking behavior; 14. Crowd of people: real-time monitoring and statistics of people in the video area, data collection, and timely warning when there are crowds of people; 15. Safety distance monitoring: The use of laser rangefinder can form three-dimensional laser color point cloud data, and timely alarm of physical objects in dangerous areas.

[0077] Step 102: Generate corresponding warning information according to the image recognition result and the image to be recognized, and transmit the warning information to a preset terminal.

[0078] In the embodiments of the present disclosure, the warning information includes but is not limited to: the image recognition result, the image to be recognized, and the personnel, equipment, time, space, environment and other information corresponding to the image to be recognized. The preset terminal is a custom-set terminal, such as a computer, a mobile phone, a large-screen terminal, etc., which can send risk warning information (warning information) to the preset terminal in real time. After receiving the warning information, the manager of the preset terminal can urge the relevant personnel on site to rectify the work process.

[0079] It should be noted that the trained image recognition model described in the embodiment of the present disclosure is placed in an edge AI analysis device. By integrating the trained image recognition model into the edge AI analysis device, it can interact with on-site monitoring equipment and network equipment to realize automatic identification of violations and immediate warning.

[0080] The preset terminals can form a smart safety supervision platform. The research and development of the smart safety supervision platform can realize functions such as intelligent equipment management, risk alarm management, video surveillance, etc. It can be connected with existing unified safety platforms and other systems, and the video surveillance data can be associated with work tasks and personnel information to achieve all-round intelligent verification of personnel, equipment, time, space, environment, and safety measures. When non-conformities are found, an alarm can be automatically issued to assist safety management personnel in fully grasping the situation at the work site.

[0081] The edge AI analysis device and the smart safety supervision platform can realize the equipment on-site self-organizing network mode (that is, the image recognition results are transmitted to the preset terminal through the preset signal device, wherein the preset signal device is used to realize the equipment on-site self-organizing network mode in the target area). For limited space and field construction scenarios, a signal transmitting end device is set at a location with good network near the work point (for example: a booster station), and a portable signal receiving device is installed at the work point to amplify and extend the wireless network signal, thereby ensuring that the image recognition results of the edge AI analysis device are transmitted to the monitoring center (the smart safety supervision platform is the preset terminal) in real time.

[0082] The security control method based on deep learning provided by the present disclosure obtains the image to be identified corresponding to the target area, and inputs the image to be identified into the trained image recognition model for image recognition processing to obtain the image recognition result; wherein the image to be identified includes at least the first monitoring target and the second monitoring target; generates corresponding warning information according to the image recognition result and the image to be identified, and transmits the warning information to the preset terminal. Compared with the related art, the embodiment of the present disclosure analyzes the image obtained on site through the trained image recognition model, generates warning information according to the analysis result (image recognition result), and transmits the warning information to the portable device or the intelligent management platform (preset terminal), which can realize the intelligent recognition and automatic alarm of the monitoring video of the work site, can automatically perform security control on multiple work points, and does not need to be controlled on site by safety supervisors, which can improve the efficiency and accuracy of security control.

[0083] In one possible implementation of the embodiment of the present disclosure, when generating the warning information, the embodiment of the present disclosure provides a schematic diagram of the generation process of the warning information, such as Figure 2 As shown, including:

[0084] Step 201: When it is determined according to the image recognition result that the first monitoring target does not meet the first preset condition, a first warning information is generated.

[0085] In the embodiment of the present disclosure, the first preset condition is a custom-set condition, and the first preset condition is mainly a constraint condition for the staff, for example: need to wear a safety helmet, need to dress in a standardized manner, need to wear a safety belt at high altitudes, and no crowds of people are allowed, etc. Specifically, the embodiment of the present disclosure does not limit the content of the first preset condition.

[0086] The first warning information is warning information related to the staff, such as: failure to place identification signs in a standardized manner, failure to wear safety helmets, failure to dress in a standardized manner, etc.

[0087] Step 202: When it is determined according to the image recognition result that the second monitoring target does not meet the second preset condition, a second warning message is generated and an alarm is issued through a preset alarm device.

[0088] In the embodiment of the present disclosure, the second preset condition is a custom-set condition, which is mainly a constraint condition for various equipment and devices in the target area, such as smoke and fire detection, leakage detection, etc. The second warning information is information related to various equipment and devices in the target area, such as smoke and flames generated at the work site, leakage of important equipment, etc. Specifically, the embodiment of the present disclosure does not limit the second preset condition and the second warning information.

[0089] At the same time, since problems occur with various equipment and devices in the target area, timely alarm processing is required to avoid greater safety hazards, so the alarm can be directly issued through the preset alarm device.

[0090] In one possible implementation of the disclosed embodiment, before automated security supervision is performed, model training needs to be performed in advance to obtain a trained image recognition model. The acquisition of the trained image recognition model may be performed by, but is not limited to, the following methods: obtaining training image data; wherein the training image data includes training images to be recognized and label data; inputting the training images to be recognized into the image recognition model for image recognition processing to obtain a training image recognition result; optimizing the image recognition model through a preset loss function based on the training image recognition result and the label data to obtain the trained image recognition model. Obtain the first target information of the first monitoring target and the second target information of the second monitoring target, and configure the first target information and the second target information in the trained image recognition model.

[0091] In the embodiments of the present disclosure, the preset loss function is a custom-set loss function, such as: mean square error loss function, cross entropy loss function, etc. The first target information is information related to the staff, such as: personnel, time, place, safety measures and other factor information, staff face photos, face verification punch-in information, etc. The second target information is other information except information related to the staff, such as: number, type, name and other information of smart devices such as safety helmets. Specifically, the embodiments of the present disclosure do not limit the preset loss function, the first target information and the second target information.

[0092] In one implementable method of the embodiment of the present disclosure, when performing image recognition on the image to be recognized, the following method may also be used but is not limited to: based on the first target information and the second target information, image recognition processing is performed on the image to be recognized through the trained image recognition model to obtain the image recognition result.

[0093] In one implementable method of the embodiment of the present disclosure, after the warning information is transmitted to the preset terminal, monitoring and management will be performed based on the warning information. Therefore, the following methods may also be used but are not limited to: visualizing the warning information; in response to a triggered communication instruction, communicating and processing with the first monitoring target through the preset terminal based on the warning information.

[0094] In the disclosed embodiment, the existing video surveillance equipment in the target area is utilized, and an edge AI analysis device is deployed to continuously analyze the existing monitoring area in the target area, so as to realize intelligent identification and automatic alarm of risk hazards such as leakage, fire hazard, and personnel violation. The found violation pictures (pictures to be identified that do not meet the preset conditions) and alarm data (early warning information) are transmitted back to the unified production safety management platform and other systems (preset terminals) through the standard integrated interface, and the management personnel receive the alarm and handle it in time.

[0095] Corresponding to the above-mentioned security management and control method based on deep learning, the embodiment of the present disclosure also provides a security management and control system based on deep learning. Figure 3 A schematic diagram of the structure of a deep learning-based security management and control system provided in an embodiment of the present disclosure.

[0096] like Figure 3 As shown, the system includes: an acquisition device 31, an analysis device 32, and a preset terminal 33.

[0097] The acquisition device 31 is used to acquire the image to be identified corresponding to the target area, and send the image to be identified to the analysis device 32;

[0098] The analysis device 32 is used to input the image to be identified into the trained image recognition model to perform image recognition processing to obtain an image recognition result; wherein the image to be identified includes at least a first monitoring target and a second monitoring target;

[0099] The analysis device 32 is further configured to generate corresponding warning information according to the image recognition result and the image to be recognized, and transmit the warning information to the preset terminal 33 .

[0100] In the disclosed embodiments, the trained image recognition model can be based on the video images of on-site personnel operations, and by simulating the working mode of biological visual systems, effectively extract the characteristics of personnel violations from the images, and accurately identify and timely warn the behaviors that appear during on-site operations, such as failure to place identification signs in a standardized manner, failure to wear safety helmets, failure to dress in a standardized manner, failure to wear safety belts at high altitudes, unmanned escalators, failure to wear insulating gloves, failure to wear insulating shoes, intrusion into prohibited areas, and overcrowding of areas. At the same time, the accuracy of the algorithm model is studied indoors, outdoors, during the day, at night, and in various weather conditions to reduce false alarms and missed reports. In order to broaden the scope of identification of violations, research is conducted on the combination of image recognition with work tickets, IoT perception, and on-site environment, continuously improve recognition accuracy, reduce the probability of on-site operation risk accidents, and ensure personnel safety.

[0101] The analysis device 32 is a developed edge AI analysis device, which integrates the trained image recognition model into the edge AI analysis device, can interact with on-site monitoring equipment and network equipment, ensure that the power consumption and energy efficiency of the device in the on-site environment meet the requirements, and has API interface, graphical interface, back-end service and other capabilities. Figure 4 As shown, Figure 4 A schematic diagram of an edge AI analysis device provided for an embodiment of the present disclosure, wherein the edge AI analysis device has its own networking function, can support 5G / WIFI, and can provide Internet access services for connected mobile control balls, smart safety helmets, etc.; the device can have its own video storage module, which can store video data of on-site operations in the edge computing terminal; the device should have long battery life; the device can have its own playback function, and remote shouting can be carried out through a mobile app.

[0102] The preset terminal 33 is the smart safety supervision platform, which can be connected to the existing unified production safety management platform to associate video content with work tasks, personnel information, etc., to achieve two-way data exchange. A two-way association relationship between monitoring video and work content can be established to achieve all-round intelligent verification of personnel, equipment, time, space, environment, and safety measures. When non-conformities are found, an alarm can be automatically sent to the unified production safety management platform. Specifically, the smart safety supervision platform can be used to Figure 5 As shown, Figure 5 A schematic diagram of a smart safety supervision platform provided for an embodiment of the present disclosure, wherein the software part should include three parts: PC application, mobile APP application, and large-screen visualization. The data of the three parts are interoperable, and the APP supports installation on at least mainstream operating systems such as Android, Hongmeng, and IOS.

[0103] The functions of the PC terminal include but are not limited to:

[0104] ① Smart operation management: In order to avoid the staff from repeatedly entering data in two systems, the system supports the import function. It can export PDF files from the existing unified safety production management platform and import them into the platform with one click. The platform can automatically extract elements such as personnel, time, location, and safety measures to support the development of on-site safety management and control. ② Smart equipment management: It can realize the management and registration management of the number, type, name, etc. of smart equipment such as safety helmets. ③ Risk alarm management: It can realize on-site AI recognition smart alarm management, including viewing alarm pictures, confirmation of various violations, single deletion, batch deletion, and recovery of alarms. ④ Video monitoring: It can realize remote real-time query of video data of multiple operation sites, and has the function of video connection and shouting intercom with on-site smart safety helmets. So that after the risk alarm, timely contact with the site can be made to eliminate accident hazards. The video clarity must meet the use requirements. ⑤ Smart alarm: It can realize real-time query of safety violation identification alarms at multiple operation sites, and can query violation types, violation time, violation pictures, etc. ⑥Data statistics: comprehensive statistics of multiple work sites’ work units, workers, violation alarms, etc. can be realized.

[0105] APP functions include but are not limited to:

[0106] ① Personnel identity verification: Use mobile phone APP to take photos of workers' faces on site, and the system can automatically check the identity and qualification information of workers. ② Smart operation management: It can support the photo taking and uploading of paper documents of work tickets, so that personnel at all levels can query online. It can support the uploading of photos of safety measures of work tickets, work permits, work termination and other on-site process operations, so that personnel at all levels can grasp the progress of the site in a timely manner. Key links such as work permits and terminations require face verification before operations are allowed. It can receive risk warning information from edge computing devices in real time, and urge relevant personnel to rectify the operation process on site. Managers can use mobile APP to talk and video with on-site personnel on smart helmets to eliminate potential accidents in time. ③ Punch in and sign in: The person in charge of construction can use mobile APP to upload the location of the work site. Key personnel can use APP to realize the punch in and sign in function based on positioning and face verification. Including construction team leaders, safety officers, safety supervision related personnel, etc. ④ Smart equipment management: It can realize the binding of on-site control balls, edge computing devices, smart helmets and other equipment, and support number and status query. ⑤Risk alarm management: You can check the AI ​​alarm information of smart devices in time on the mobile APP, including time, pictures, and related operations. For managers and safety officers, you can confirm, delete single items, delete in batches, and restore alarms for various violations. ⑥Smart intercom: It can support real-time intercom calls between all management and operating personnel during the operation process, and also support shouting to the on-site edge workstations. ⑦ Daily work report: It can support online viewing of daily task work report documents on the mobile phone.

[0107] The functions of the large screen include but are not limited to: it can support real-time viewing of on-site operation videos, violations, etc. on the large screen. Violations can be displayed in real time on the large screen and mobile phone, and classified statistical analysis can be performed in the background.

[0108] At the same time, the edge AI analysis device and the smart safety supervision platform can realize the on-site self-organizing network mode of the equipment (that is, the image recognition results are transmitted to the preset terminal through the preset signal device, wherein the preset signal device is used to realize the on-site self-organizing network mode of the equipment in the target area). For limited space and field construction scenes, a signal transmitting terminal device is set at a location with a good network near the operation point (for example: a booster station), and a portable signal receiving device is installed at the operation point to amplify and extend the wireless network signal, so as to ensure that the image recognition results of the edge AI analysis device are transmitted to the monitoring center (the smart safety supervision platform is the preset terminal) in real time. Specifically, Figure 6 As shown, Figure 6 A schematic diagram of a networking mode provided in an embodiment of the present disclosure.

[0109] In an implementation manner of the embodiment of the present disclosure, the preset terminal 33 includes: a first terminal 331, a second terminal 332 and a third terminal 333.

[0110] The first terminal 331 and the second terminal 332 are used to obtain first target information of the first monitoring target and second target information of the second monitoring target;

[0111] The first terminal 331 and the second terminal 332 are also used to visualize the warning information;

[0112] The first terminal 331 and the second terminal 332 are also used to communicate with the first monitoring target through the preset terminal based on the warning information in response to the triggered communication instruction;

[0113] The third terminal 333 is used to visualize the warning information.

[0114] Among them, the first terminal is the PC terminal, the second terminal is the APP terminal, and the third terminal is the large-screen terminal.

[0115] Corresponding to the above-mentioned security control method based on deep learning, the present invention also proposes a security control device based on deep learning. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, the details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be repeated in the present invention.

[0116] Figure 7A schematic diagram of a deep learning-based security management and control device provided in an embodiment of the present disclosure is shown in FIG. Figure 7 As shown, including:

[0117] An acquisition unit 71 is used to acquire an image to be recognized corresponding to a target area;

[0118] The processing unit 72 is used to input the image to be identified into the trained image recognition model for image recognition processing to obtain an image recognition result; wherein the image to be identified includes at least the first monitoring target and the second monitoring target;

[0119] A generating unit 73, configured to generate corresponding warning information according to the image recognition result and the image to be recognized;

[0120] The transmission unit 74 is used to transmit the warning information to a preset terminal.

[0121] The security control device based on deep learning provided by the present disclosure obtains the image to be identified corresponding to the target area, and inputs the image to be identified into the trained image recognition model for image recognition processing to obtain the image recognition result; wherein the image to be identified includes at least the first monitoring target and the second monitoring target; generates corresponding warning information according to the image recognition result and the image to be identified, and transmits the warning information to the preset terminal. Compared with the related art, the embodiment of the present disclosure analyzes the image obtained on site through the trained image recognition model, generates warning information according to the analysis result (image recognition result), and transmits the warning information to the portable device or the intelligent management platform (preset terminal), which can realize the intelligent recognition and automatic alarm of the monitoring video of the work site, can automatically perform safety control on multiple work points, and does not need to be controlled on site by safety supervisors, which can improve the efficiency and accuracy of safety control.

[0122] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Figure 8 As shown, the generating unit 73 includes:

[0123] A generating module 731 is used to generate a first warning message when it is determined according to the image recognition result that the first monitoring target does not meet the first preset condition;

[0124] The generating module 731 is further configured to generate second warning information when it is determined according to the image recognition result that the second monitoring target does not meet the second preset condition;

[0125] The alarm module 732 is used to issue an alarm through a preset alarm device.

[0126] Furthermore, in a possible implementation of the embodiment of the present disclosure, the acquisition unit 71 is further used to acquire training image data; wherein the training image data includes training images to be recognized and label data;

[0127] The processing unit 72 is also used to input the training image to be recognized into the image recognition model to perform image recognition processing to obtain a training image recognition result;

[0128] like Figure 8 As shown, the device also includes:

[0129] The optimization unit 75 is used to optimize the image recognition model through a preset loss function according to the training image recognition result and the label data to obtain the trained image recognition model.

[0130] The acquisition unit 71 is further used to acquire first target information of the first monitoring target and second target information of the second monitoring target;

[0131] The configuration unit 76 is used to configure the first target information and the second target information in the trained image recognition model.

[0132] Furthermore, in a possible implementation of the embodiment of the present disclosure, the processing unit 72 is also used to perform image recognition processing on the image to be recognized through the trained image recognition model based on the first target information and the second target information to obtain the image recognition result.

[0133] Furthermore, in a possible implementation of the embodiment of the present disclosure, as Figure 8 As shown, the device also includes:

[0134] A display unit 77, used for visualizing the warning information;

[0135] The communication unit 78 is used to respond to the triggered communication instruction and communicate with the first monitoring target through the preset terminal based on the early warning information.

[0136] It should be noted that the above explanation of the method embodiment is also applicable to the device of the embodiment of the present disclosure, and the principle is the same, which is no longer limited in the embodiment of the present disclosure.

[0137] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0138] Fig. 9A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0139] like Fig. 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 902 or a computer program loaded from a storage unit 908 to a RAM (Random Access Memory) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An I / O (Input / Output) interface 905 is also connected to the bus 904.

[0140] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0141] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as a security management method based on deep learning. For example, in some embodiments, a security management method based on deep learning may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the aforementioned deep learning-based security management method in any other appropriate manner (for example, by means of firmware).

[0142] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a special purpose or general purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0143] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0144] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an EPROM (Electrically Programmable Read-Only-Memory) or a flash memory, an optical fiber, a CD-ROM (Compact Dis sc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0145] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0146] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0147] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0148] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0149] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0150] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A security management and control method based on deep learning, characterized in that: include: Obtain an image to be identified corresponding to the target area, and input the image to be identified into a trained image recognition model for image recognition processing to obtain an image recognition result; wherein the image to be identified includes at least a first monitoring target and a second monitoring target; Generate corresponding warning information according to the image recognition result and the image to be recognized, and transmit the warning information to a preset terminal.

2. The method according to claim 1, characterized in that The generating corresponding warning information according to the image recognition result and the image to be recognized includes: When it is determined according to the image recognition result that the first monitoring target does not meet the first preset condition, generating a first warning information; When it is determined according to the image recognition result that the second monitoring target does not meet the second preset condition, a second warning information is generated and an alarm is issued through a preset alarm device.

3. The method according to claim 1, characterized in that Before acquiring the image to be recognized corresponding to the target area, the method further includes: Acquire training image data; wherein the training image data includes training images to be identified and label data; Inputting the training image to be recognized into the image recognition model for image recognition processing to obtain a training image recognition result; According to the training image recognition result and the label data, the image recognition model is optimized by a preset loss function to obtain the trained image recognition model. Acquire first target information of the first monitoring target and second target information of the second monitoring target, and configure the first target information and the second target information in the trained image recognition model.

4. The method according to claim 1, characterized in that: The step of inputting the image to be recognized into a trained image recognition model for image recognition processing to obtain an image recognition result comprises: Based on the first target information and the second target information, the image to be recognized is subjected to image recognition processing by the trained image recognition model to obtain the image recognition result.

5. The method according to claim 1, characterized in that After transmitting the warning information to a preset terminal, the method further includes: Visualizing the warning information; In response to the triggered communication instruction, communication processing is performed with the first monitoring target through the preset terminal based on the early warning information.

6. A security management and control system based on deep learning, characterized in that: include: Acquisition device, analysis device, preset terminal, The acquisition device is used to acquire the image to be identified corresponding to the target area, and send the image to be identified to the analysis device; The analysis device is used to input the image to be identified into a trained image recognition model to perform image recognition processing to obtain an image recognition result; wherein the image to be identified includes at least a first monitoring target and a second monitoring target; The analysis device is also used to generate corresponding warning information according to the image recognition result and the image to be recognized, and transmit the warning information to the preset terminal.

7. The system according to claim 6, characterized in that The preset terminals include: a first terminal, a second terminal and a third terminal. The first terminal and the second terminal are used to obtain first target information of the first monitoring target and second target information of the second monitoring target; The first terminal and the second terminal are further used to visualize the warning information; The first terminal and the second terminal are also used to, in response to the triggered communication instruction, communicate with the first monitoring target through the preset terminal based on the warning information; The third terminal is used to visualize the warning information.

8. A security control device based on deep learning, characterized in that: include: An acquisition unit, used for acquiring an image to be identified corresponding to a target area; A processing unit, used for inputting the image to be identified into a trained image recognition model for image recognition processing to obtain an image recognition result; wherein the image to be identified includes at least a first monitoring target and a second monitoring target; A generating unit, configured to generate corresponding warning information according to the image recognition result and the image to be recognized; The transmission unit is used to transmit the warning information to a preset terminal.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

Images