Vision and light wave combined space monitoring method and system

Abstract

The invention relates to a vision and light wave combined space monitoring method and system, and the method comprises the steps: calling a three-dimensional point cloud calculation function and an image recognition function, and carrying out the analysis of received visual image data and echo data. The analysis of the echo data can determine that foreign matters intrude into the light wave monitoring area. After the foreign matter intrudes into the light wave monitoring area, the situation that a person does not wear a safety helmet can be determined through analysis of visual image data. The current alarm information is displayed, and the latest alarm information content is displayed and is composed of the alarm content and the alarm time. Through a space monitoring method combining vision and light waves, the method can be timely and effectively used in a complex space obstacle positioning scene. The device can simply display light wave alarm information, detect whether foreign matters intrude in, detect whether people fall down on site, detect whether people smoke on site, detect whether people do not wear work clothes according to regulations, and detect whether people on site wear safety helmets according to regulations.

Description

Technical Field

[0001] The present application relates to the field of space monitoring technology, and in particular to a space monitoring method and system combining vision and light waves. Background Art

[0002] The three-dimensional point cloud spatial positioning technology based on the light wave radio frequency method has a wide range of applications in the field of spatial positioning. Since the three-dimensional point cloud spatial positioning technology uses a relatively complex echo analysis technology, it is difficult for the traditional three-dimensional point cloud spatial positioning technology to be used effectively and timely in the positioning scene of complex spatial obstacles. Whether there are foreign objects in the monitoring area, whether there are people falling, whether there are people smoking, whether there are people not wearing work clothes as required, and whether they are wearing safety helmets as required are key monitoring projects. Therefore, it is necessary to propose a spatial monitoring method and system combining vision and light waves to address the defect that the three-dimensional point cloud spatial positioning technology based on the traditional light wave radio frequency method is difficult to be used in a timely and effective manner in the positioning scene of complex spatial obstacles. Summary of the invention

[0003] Based on this, it is necessary to propose a spatial monitoring method and system that combines vision and light waves to address the defect that the three-dimensional point cloud spatial positioning technology of the traditional light wave radio frequency method is difficult to use in a timely and effective manner in the positioning scenes of complex spatial obstacles.

[0004] The present application provides a space monitoring method combining vision and light waves, comprising: Call 3D point cloud computing function and image recognition function; Receive visual image data and superimpose echo data; Based on the three-dimensional point cloud computing function, the superimposed echo data is calculated to obtain the echo characteristic data; Use echo feature data to determine whether an entity has entered the monitored space; If no entity enters the monitored space, the received visual image data and the superimposed echo data are returned until a shutdown command is received; If an entity enters the monitored space, the visual image data with the same timestamp as the echo feature data is called; Determine the danger level of the entity in the monitored space based on the image recognition function and the visual image data; Feedback the monitoring results using the danger level of the space where the entity is monitored; The received visual image data and the superimposed echo data are returned until a shutdown instruction is received.

[0005] The present application provides a space monitoring system combining vision and light waves, comprising: A monitoring body, used to execute the space monitoring method combining vision and light waves; The memory is communicatively connected with the monitoring body.

[0006] The present application relates to a spatial monitoring method and system combining vision and light waves, which analyzes the received visual image data and echo data by calling a three-dimensional point cloud computing function and an image recognition function. The analysis of the echo data can determine whether a foreign object has entered the light wave monitoring area. After determining that a foreign object has entered the light wave monitoring area, the analysis of the visual image data can determine whether the personnel are not wearing a safety helmet, etc. Then the current alarm information is displayed, and the content of the most recent alarm information is displayed, which is composed of the alarm content and the alarm time. The spatial monitoring method combining vision and light waves can be used in a timely and effective manner in complex spatial obstacle positioning scenarios. Simply display the light wave alarm information, detect whether there is a foreign object intruding, detect whether there is a person falling at the scene, detect whether there is a person smoking at the scene, detect whether there is a person not wearing work clothes as required at the scene, and detect whether the personnel at the scene are wearing safety helmets as required. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 A schematic flow chart of a space monitoring method combining vision and light waves provided in one embodiment of the present application.

[0008] Figure 2 A structural diagram of a space monitoring system combining vision and light waves provided in one embodiment of the present application.

[0009] Reference numerals: 100-monitoring body; 200-memory. DETAILED DESCRIPTION

[0010] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0011] The present application provides a space monitoring method combining vision and light waves.

[0012] like Figure 1 As shown, in one embodiment of the present application, a space monitoring method combining vision and light waves includes: S100, calling a three-dimensional point cloud computing function and an image recognition function.

[0013] Specifically, the image recognition function includes a connection function and an image calling function.

[0014] The connection function is used to connect the communication between the processor and the memory. Or the connection function is used to connect the processor and the cloud server after networking.

[0015] The image call function is used to pre-process the image, and then send the processed image to the cloud server connected to the network for analysis to obtain the danger level of the monitored space where the entity in the image is located.

[0016] The image calling function may also call the image processing model in the memory to the processor to obtain the danger level of the monitored space where the entity in the image is located.

[0017] S200, receiving visual image data and superimposing echo data.

[0018] Specifically, the visual image data may be a color image or a binary image.

[0019] The superimposed echo data includes the amplitude and frequency of the echo.

[0020] S300, based on the three-dimensional point cloud computing function, calculate and stack the echo data to obtain echo feature data.

[0021] Specifically, using the three-dimensional point cloud computing function, the basic light wave of the environment can be determined based on the superimposed echo data, the amplitude and frequency of the emitted light wave. The basic light wave of the environment corresponds to the echo feature data. The echo feature data can be used to determine whether an entity has entered the monitored space.

[0022] S400, using the echo characteristic data, determining whether an entity has entered the monitored space.

[0023] S500: If no entity enters the monitored space, the received visual image data and echo data are returned until a shutdown command is received.

[0024] S600, if an entity enters the monitored space, visual image data having the same timestamp as the echo feature data is called.

[0025] S700, determining the danger level of the entity in the monitored space based on the image recognition function and the visual image data.

[0026] S800 uses the danger level of the monitored space where the entity is located to feed back the monitoring result.

[0027] S900, returning to the received visual image data and superimposed echo data until a shutdown instruction is received.

[0028] The present embodiment relates to a spatial monitoring method and system combining vision and light waves, which analyzes the received visual image data and echo data by calling a three-dimensional point cloud computing function and an image recognition function. The analysis of the echo data can determine whether a foreign object has entered the light wave monitoring area. After determining that a foreign object has entered the light wave monitoring area, the analysis of the visual image data can determine whether the personnel are not wearing a helmet, etc. Then display the current alarm information, and display the content of the most recent alarm information, which consists of the alarm content and the alarm time. The spatial monitoring method combining vision and light waves can be used in a timely and effective manner in complex spatial obstacle positioning scenarios. Simply display the light wave alarm information, detect whether there is a foreign object entering, detect whether there is a person falling at the scene, detect whether there is a person smoking at the scene, detect whether there is a person not wearing work clothes as required at the scene, and detect whether the personnel at the scene are wearing helmets as required.

[0029] In one embodiment of the present application, in S100, the three-dimensional point cloud computing function includes: S111, determining the type of system management.

[0030] S112, constructing an instruction framework of computing instructions based on the type of system management.

[0031] S113, embedding a communication authorization module in the instruction framework.

[0032] S114, incorporating a processing script based on the communication authorization module.

[0033] Specifically, the type of system management can determine the instructions. For example, the command prompt (CMD) of some open source systems can call the communication authorization module, processing script, etc. in the instruction framework of the computing instruction. By entering commands in the CMD window, various system operations and tasks can be performed, such as creating, copying, deleting files, starting applications, configuring network settings, etc.

[0034] The communication authorization module can execute network-related commands and perform three-dimensional point cloud computing functions. The communication authorization module can improve the communication between the cloud and the space monitoring system that combines vision and light waves. It can also improve the communication capabilities of the processor and memory, thereby improving the complex light wave environment handling capabilities of the space monitoring system that combines vision and light waves.

[0035] In one embodiment of the present application, in S100, the three-dimensional point cloud computing function further includes: S121, establishing an echo feature feedback assignment table.

[0036] Simply put, when faced with complex echoes in a large environment, fuzzy algorithms can be used to determine echo feature data.

[0037] Generally speaking, if there is no foreign matter entering the monitoring area, the frequency and amplitude of the echo will be in a stable fluctuation range.

[0038] When a foreign object enters the monitoring area, the frequency and amplitude of the echo will fluctuate.

[0039] More specifically, when the frequency and amplitude of the echo are in a stable fluctuation range, the table value of the echo feature feedback assignment table is 0; when the frequency and amplitude of the echo fluctuate, the table value of the echo feature feedback assignment table is 1.

[0040] S122, establishing an echo wave equation based on the frequency and amplitude of the echo.

[0041] S123, incorporate the echo wave equation into the processing script.

[0042] S124, incorporating the echo feature feedback assignment table into the instruction framework of the calculation instruction.

[0043] Specifically, incorporating the echo wave equation into the processing script allows for simple and quick calculation of superimposed echo data.

[0044] In one embodiment of the present application, S122 includes: (Formula 1) in, is the echo after superposition, is the basic light wave of the environment, is the light wave emitted by the local device, is the environmental regulation constant, is the amplitude of the basic light wave of the environment, is the inverse of the frequency of the basic light wave of the environment, is the amplitude of the light wave emitted by the local device, is the inverse of the frequency of the light wave emitted by the local device, is the superimposed damping.

[0045] Specifically, superimposed damping is related to the attenuation rate of vibration, and the attenuation rate is often expressed as Calculated by the method.

[0046] There is a linear relationship between superposition damping and the attenuation rate of vibration with a constant coefficient. When the fuzzy algorithm is used, the constant coefficient is ignored in this embodiment, thereby improving the effectiveness of light wave analysis in complex environments.

[0047] In fact, in the working state of large monitoring data volume, due to the superposition between light waves, the superposition damping is small and approaches zero, so the superposition damping can take a non-zero constant. When the superposition damping takes a constant less than 1, the high-frequency light in the basic light wave of the stronger environment can be adjusted.

[0048] When the superposition damping takes a constant greater than 1, the low-frequency light in the fundamental light wave of the weaker environment can be adjusted.

[0049] In one embodiment of the present application, in S100, the image recognition function includes: S131, based on the cross-platform computer vision library, generate a cross-platform computer vision library calling function.

[0050] S132, using the type managed by the system, generate a connection function.

[0051] S133, incorporating the cross-platform computer vision library calling function and connection function into the image recognition function.

[0052] Specifically, OpenCV (open source computer vision library) is a cross-platform computer vision library based on BSD open source, which can run on Linux, Windows, Android and Mac OS operating systems. It is lightweight and efficient - it consists of a series of C functions and a small number of C++ classes, and provides interfaces for languages ​​such as Python, Ruby, and MATLAB, and implements many common algorithms in image processing and computer vision. OpenCV is written in C++, and its main interface is also in C++, but it still retains a large number of C language interfaces. In the development of computer vision projects, OpenCV, as a more popular open source library, has a rich library of commonly used image processing functions. It is written in C / C++ and can run on operating systems such as Linux / Windows / Mac, and can quickly implement some image processing and recognition tasks. OpenCV also provides interfaces for Java, python, cuda, etc., and basic algorithm calls for machine learning.

[0053] As a part of the image recognition function, the connection function can connect the operation program of the processor and the storage calling program of the memory.

[0054] The connection function can also connect the processor's computing program and the cloud storage calling program, and the connection function can connect the processor's computing program and the cloud computing program, thereby realizing the processor's high applicability in complex optical wave parsing and analysis environments.

[0055] In one embodiment of the present application, S300 includes: S311, receiving the superimposed echo data.

[0056] S312: Based on the amplitude and frequency in the superimposed echo data, determine the amplitude and frequency of the light wave emitted by the local device corresponding to the echo data.

[0057] S313, using the amplitude and frequency in the superimposed echo data and the amplitude and frequency of the light wave emitted by the local device, calculate the amplitude and frequency of the basic light wave of the environment.

[0058] S314, defining the amplitude and frequency of the basic light wave of the environment as echo data.

[0059] S321, call the echo feature feedback assignment table.

[0060] S322, searching the echo characteristic feedback value table according to the amplitude and frequency of the echo data.

[0061] S323, determine echo characteristic data.

[0062] Simply put, when a foreign object appears in the monitoring area, such as a person entering the monitoring area, the light wave emitted by the local device will produce an echo, and by receiving this superimposed echo, the basic light wave of the environment can be analyzed.

[0063] Under normal circumstances, the basic light waves of the environment will not fluctuate. When people enter the monitoring area, the basic light waves of the environment will fluctuate.

[0064] At this time, after the echo data is queried in the echo characteristic feedback assignment table, it can be determined that the echo characteristic data is 1.

[0065] Furthermore, when no foreign matter enters the monitoring area, the echo characteristic data can be determined to be 0 after searching the echo characteristic feedback value table.

[0066] In one embodiment of the present application, S400 includes: S410, receiving echo characteristic data.

[0067] S420, determining whether the echo characteristic data fluctuates.

[0068] S430: If there is no fluctuation in the echo characteristic data, it is determined that no entity has entered the monitored space.

[0069] S440: If the echo characteristic data fluctuates, it is determined that an entity has entered the monitored space.

[0070] Simply put, if the echo characteristic data is 1, it is determined that an entity has entered the monitored space.

[0071] The echo characteristic data is 0, confirming that no entity has entered the monitored space.

[0072] When it is determined that an entity has entered the monitored space, the real-time image can be incorporated into the image recognition function for calculation.

[0073] When no entity enters the monitored space, the basic light wave monitoring program of the environment can continue to be executed without the intervention of the image recognition function.

[0074] In one embodiment of the present application, S700 includes: S710, obtaining a mark value output by an image recognition function according to the visual image data.

[0075] S720, calling a numerical risk table.

[0076] S730: Based on the flag value and the value risk table, determine whether the flag value matches the smoking alarm value.

[0077] S731, if the flag value matches the smoking alarm value, assign the danger level of the smoking alarm to the visual image data.

[0078] S732: If the value of the flag does not match the value of the smoking alarm, determine whether the value of the flag matches the value of the fall alarm.

[0079] S733: If the value of the flag matches the value of the fall alarm, the danger level of the fall alarm is assigned to the visual image data.

[0080] S734, if the value of the flag does not match the value of the fall alarm, determine whether the value of the flag matches the value of the helmet alarm.

[0081] S735, if the value of the sign matches the value of the helmet warning, the danger level of the helmet warning is assigned to the visual image data.

[0082] S736, if the value of the sign does not match the value of the helmet warning, determine whether the value of the sign matches the value of the work clothes warning.

[0083] S737, if the value of the sign matches the value of the work clothes alarm, the danger level of the work clothes alarm is assigned to the visual image data.

[0084] S738: If the value of the mark does not match the value of the work clothes alarm, the information of data processing abnormality is fed back.

[0085] Specifically, a surveillance video display shows real-time camera capture data.

[0086] Light wave data display shows the light wave arrival data.

[0087] The system will start working and monitor the area where we deploy it to detect whether any foreign objects break in.

[0088] The work content is: Light wave alarm: Display light wave alarm information to detect whether there is any foreign object intrusion.

[0089] Fall alarm: Detect whether there is a fall at the scene.

[0090] Smoking alarm: Detect whether there is anyone smoking at the scene.

[0091] Work clothes alarm: Check whether there are people on site who are not wearing work clothes as required.

[0092] Safety helmet alarm: Check whether on-site personnel are wearing safety helmets as required.

[0093] Check the alarm information of a certain time and see the alarm display at the corresponding moment. If it is a light wave alarm information, you can see the position where the light wave alarm information is projected into the picture.

[0094] The present application provides a space monitoring system combining vision and light waves.

[0095] like Figure 2 As shown, in one embodiment of the present application, a space monitoring system combining vision and light waves includes a monitoring body 100 and a memory 200.

[0096] The monitoring body 100 is used to execute the spatial monitoring method combining vision and light waves.

[0097] The memory 200 is in communication connection with the monitoring body 100 .

[0098] Specifically, the monitoring body 100 includes a light wave transmitter, a light wave receiver, and a video recorder.

[0099] The present embodiment relates to a space monitoring system combining vision and light waves. The monitoring body 100 calls the three-dimensional point cloud computing function and image recognition function of the memory 200, or the three-dimensional point cloud computing function calculation link and the image recognition function calculation link. The received visual image data and echo data are analyzed. The analysis of the echo data can determine that there is a foreign object intruding into the light wave monitoring area. After determining that there is a foreign object intruding into the light wave monitoring area, the analysis of the visual image data can determine whether the personnel are not wearing a helmet. Then display the current alarm information, display the content of the most recent alarm information, which consists of the alarm content and the alarm time. Through the space monitoring method combining vision and light waves, it can be used in a timely and effective manner in the positioning scene of complex space obstacles. Simply display the light wave alarm information, detect whether there is a foreign object intruding, detect whether there is a person falling at the scene, detect whether there is a person smoking at the scene, detect whether there is a person not wearing work clothes as required at the scene, and detect whether the personnel at the scene wear a helmet as required.

[0100] The technical features of the above-described embodiments may be arbitrarily combined, and the execution order of the method steps is not limited. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0101] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A space monitoring method combining vision and light waves, characterized in that: include: Call 3D point cloud computing function and image recognition function; Receive visual image data and superimpose echo data; Based on the three-dimensional point cloud computing function, the superimposed echo data is calculated to obtain the echo characteristic data; Use echo feature data to determine whether an entity has entered the monitored space; If no entity enters the monitored space, the received visual image data and the superimposed echo data are returned until a shutdown command is received; If an entity enters the monitored space, the visual image data with the same timestamp as the echo feature data is called; Determine the danger level of the entity in the monitored space based on the image recognition function and the visual image data; Feedback the monitoring results using the danger level of the space where the entity is monitored; The received visual image data and the superimposed echo data are returned until a shutdown instruction is received.

2. The space monitoring method combining vision and light waves according to claim 1 is characterized in that: In the calling of the three-dimensional point cloud computing function and the image recognition function, the three-dimensional point cloud computing function includes: Determine the type of system administration; Based on the type of system management, an instruction framework for computing instructions is constructed; Embedding a communication authorization module in the instruction framework; Based on the communication authorization module, it is incorporated into the processing script.

3. The space monitoring method combining vision and light waves according to claim 2 is characterized in that: The three-dimensional point cloud computing function also includes: Establishing echo characteristic feedback assignment table; Based on the frequency and amplitude of the echo, the echo wave equation is established; Incorporate the echo wave equation into the processing script; The echo characteristic feedback assignment table is incorporated into the instruction framework of the calculation instructions.

4. The space monitoring method combining vision and light waves according to claim 3 is characterized in that: The echo wave equation is established based on the frequency and amplitude of the echo, including: in, is the echo after superposition, is the basic light wave of the environment, is the light wave emitted by the local device, is the environmental regulation constant, is the amplitude of the basic light wave of the environment, is the inverse of the frequency of the basic light wave of the environment, is the amplitude of the light wave emitted by the local device, is the inverse of the frequency of the light wave emitted by the local device, is the superimposed damping.

5. The space monitoring method combining vision and light waves according to claim 4 is characterized in that: In the calling of the three-dimensional point cloud computing function and the image recognition function, the image recognition function includes: Based on the cross-platform computer vision library, generate a cross-platform computer vision library call function; Generate connection functions using system-managed types; Incorporate cross-platform computer vision library calling functions and connection functions into image recognition functions.

6. The space monitoring method combining vision and light waves according to claim 5 is characterized in that: The computing of the echo data based on the three-dimensional point cloud computing function to obtain the echo characteristic data includes: receiving the superimposed echo data; Determine the amplitude and frequency of the light wave emitted by the local device corresponding to the echo data based on the amplitude and frequency in the superimposed echo data; The amplitude and frequency of the basic light waves of the environment are calculated by using the amplitude and frequency in the superimposed echo data and the amplitude and frequency of the light waves emitted by the local device; The amplitude and frequency of the fundamental light waves that define the environment are the echo data.

7. The space monitoring method combining vision and light waves according to claim 6 is characterized in that: The method of calculating the echo data based on the three-dimensional point cloud computing function to obtain the echo characteristic data also includes: Call the echo feature feedback assignment table; According to the amplitude and frequency of the echo data, the echo characteristic feedback assignment table is searched; Determine echo characteristic data.

8. The space monitoring method combining vision and light waves according to claim 7 is characterized in that: The method of using the echo characteristic data to determine whether an entity has entered the monitored space includes: receiving echo characteristic data; Determine whether the echo characteristic data fluctuates; If the echo characteristic data does not fluctuate, it is determined that no entity has entered the monitored space; If the echo characteristic data fluctuates, it is determined that an entity has entered the monitored space.

9. The space monitoring method combining vision and light waves according to claim 8 is characterized in that: The step of determining the danger level of the entity in the monitored space based on the image recognition function and the visual image data includes: According to the visual image data, a sign value output by the image recognition function is obtained; Call the numerical risk table; Based on the marker value and the numerical risk table, determining whether the marker value matches the smoking warning value; If the value of the flag matches the value of the smoking warning, the danger level of the smoking warning is assigned to the visual image data; If the value of the flag does not match the value of the smoking alarm, then determining whether the value of the flag matches the value of the fall alarm; If the value of the flag matches the value of the fall warning, the danger level of the fall warning is assigned to the visual image data; If the value of the sign does not match the value of the fall alarm, then determine whether the value of the sign matches the value of the helmet alarm; If the value of the sign matches the value of the helmet warning, the danger level of the helmet warning is assigned to the visual image data; If the value of the sign does not match the value of the helmet warning, then determine whether the value of the sign matches the value of the work clothes warning; If the value of the sign matches the value of the work clothes warning, the danger level of the work clothes warning is assigned to the visual image data; If the value of the mark does not match the value of the work clothes alarm, the information of data processing abnormality is fed back.

10. A space monitoring system combining vision and light waves, characterized in that: include: A monitoring body, used to perform the space monitoring method combining vision and light waves as claimed in any one of claims 1 to 9; The memory is communicatively connected with the monitoring body.

Images