Security protection method, electronic device, storage medium and computer program

Abstract

The application discloses a safety protection method, an electronic device, a storage medium and a computer program, and relates to the technical field of machine vision, and comprises the following steps: acquiring an image of a moving object in a to-be-detected area; determining a horizontal coordinate, a vertical coordinate and a measurement angle of the moving object according to the image of the moving object, and matching a horizontal distortion rate and a vertical distortion rate of the measurement angle from a preset mapping relationship; determining a corrected horizontal coordinate and a corrected vertical coordinate according to the horizontal coordinate, the vertical coordinate, the horizontal distortion rate and the vertical distortion rate of the measurement angle, and performing safety protection according to the corrected horizontal coordinate and the corrected vertical coordinate. Therefore, through distortion rate matching and coordinate correction, the problem that, when a fisheye lens is imaged, the image of a moving object is distorted, leading to inaccurate image-based coordinate positioning and safety protection, is solved, the accuracy of position judgment of a moving object is greatly improved, and the effectiveness of safety protection measures is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, and in particular to a safety protection method, an electronic device, a storage medium and a computer program. BACKGROUND

[0002] With the deepening of industrial automation, the wide application of automatic equipment in the production line greatly improves the production efficiency, but the potential collision risk between high-speed operation of equipment and personnel and foreign matters makes real-time monitoring and safety protection of the detection area a key link of industrial safety.

[0003] Currently, the industry adopts industrial lenses or fisheye lenses to collect images, detects moving objects through machine vision algorithms, extracts coordinates, and triggers protection actions in combination with preset regions. Some solutions set shielding areas for the moving parts of the equipment to reduce false positives.

[0004] However, the distortion problem of fisheye lenses leads to insufficient positioning accuracy of moving objects, and there is a lack of standardized distortion correction logic. The division of dangerous areas and warning areas often deviates from the actual operating radius of the equipment, and the protection action triggering logic is not standardized enough, which can easily cause false triggering or missed detection, and needs to be solved urgently. SUMMARY

[0005] The present application provides a safety protection method to at least solve the problem of inaccurate coordinate positioning and safety protection based on images due to the distortion of moving object images when fisheye lenses are used, greatly improving the accuracy of position judgment of moving objects and ensuring the effectiveness of safety protection measures.

[0006] To achieve the above-mentioned purpose, the first aspect of the present application provides a safety protection method, which comprises the following steps:

[0007] obtaining an image of a moving object in a detection area;

[0008] determining the horizontal coordinate, vertical coordinate and measurement angle of the moving object according to the image of the moving object, and matching the horizontal distortion rate and vertical distortion rate of the measurement angle from a preset mapping relationship;

[0009] determining the corrected horizontal coordinate and corrected vertical coordinate according to the horizontal coordinate, vertical coordinate, horizontal distortion rate and vertical distortion rate of the measurement angle, and performing safety protection according to the corrected horizontal coordinate and corrected vertical coordinate.

[0010] According to the security protection method provided in the embodiments of the present application, the horizontal and vertical coordinates and the measurement angle of the moving object are determined, the horizontal distortion rate and the vertical distortion rate are matched from the preset mapping relationship, the horizontal and vertical coordinates of the distortion are corrected, the problem that the image of the moving object is distorted when the fisheye lens is imaged, and the image-based coordinate positioning and security protection are inaccurate is solved, the accuracy of the position judgment of the moving object is greatly improved, and the effectiveness of the security protection measures is ensured.

[0011] To achieve the above object, the second aspect of the present application further provides an electronic device, comprising: a memory for storing a computer program; a processor for executing the computer program to implement the steps of any of the above security protection methods.

[0012] To achieve the above object, the third aspect of the present application further provides a non-volatile computer readable storage medium, and the computer readable storage medium stores a computer program, wherein the computer program is executed by a processor to implement the steps of any of the above security protection methods.

[0013] To achieve the above object, the fourth aspect of the present application further provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement the steps of any of the above security protection methods.

[0014] Additional aspects and advantages of the present application will be made apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0016] Figure 1 The pillow-shaped distortion and barrel-shaped distortion schematic diagram provided for an embodiment of the present application;

[0017] Figure 2 The positive and negative distortion schematic diagram provided for an embodiment of the present application;

[0018] Figure 3 The angle lens top-down imaging physical effect diagram provided for an embodiment of the present application;

[0019] Figure 4 The positive barrel-shaped distortion schematic diagram provided for an embodiment of the present application;

[0020] Figure 5A geometric diagram of a pinhole camera imaging model provided by an embodiment of the present application;

[0021] Figure 6 A projection curve diagram of different distortion models provided by an embodiment of the present application;

[0022] Figure 7 A geometric projection model diagram of a fisheye lens provided by an embodiment of the present application;

[0023] Figure 8 A flowchart of a safety protection method provided by an embodiment of the present application;

[0024] Figure 9 A checkerboard diagram provided by an embodiment of the present application;

[0025] Figure 10 A diagram of distortion correction parameter measurement provided by an embodiment of the present application;

[0026] Figure 11 A safety region layering diagram under a camera view provided by an embodiment of the present application;

[0027] Figure 12 A dangerous-warning-safe region nesting diagram under a camera field of view provided by an embodiment of the present application;

[0028] Figure 13 A circuit board region division diagram provided by an embodiment of the present application;

[0029] Figure 14 A block diagram of a safety protection device provided by an embodiment of the present application;

[0030] Figure 15 A structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0032] It should be noted that in the description of the present application, the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or equipment. The terms "first", "second" and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0033] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below in conjunction with the drawings and specific embodiments.

[0034] Before introducing the safety protection method of the embodiment of the present application, the distortion mode of the current wide-angle fisheye lens will be briefly introduced.

[0035] The fisheye lens with wide angle can detect a larger coverage range, but because the focal length is short, the image closer to the edge is more distorted. Distortion is the phenomenon that the size ratio of the formed image and the actual object is deviated after imaging through the lens. There are usually two kinds of distortion, namely, pillow-shaped distortion and barrel-shaped distortion, as shown in Figure 1 Figure 1 The pillow-shaped distortion and barrel-shaped distortion schematic diagram provided by an embodiment of the present application, Figure 1 In (a), the pillow-shaped distortion schematic diagram is shown. The pillow-shaped distortion is that the real straight line of the object is contracted and bent towards the center of the image after imaging. The overall shape is similar to the surface profile of a pillow, which causes the size of the object at the edge of the image to be compressed, and the proportion is unbalanced with the center area. Figure 1 In (b), the barrel-shaped distortion schematic diagram is shown. The barrel-shaped distortion is that the real straight line of the object is expanded and bent towards the outside of the image. The overall shape is similar to the side profile of a barrel, which causes the size of the object at the edge of the image to be stretched, and the proportion is seriously unbalanced with the center area. The reason for the above two phenomena is that the magnification in X / Y direction is different after the light at different positions passes through the lens.

[0036] Distortion can also be divided into two categories according to other classifications, defined as positive distortion and negative distortion. Positive distortion and negative distortion are defined according to the size of the imaging and the actual object size. If the imaging size is larger than the original size, it is defined as positive distortion. If the imaging size is smaller than the original size, it is defined as negative distortion, as shown in Figure 2 Figure 2 The positive distortion and negative distortion schematic diagram provided by an embodiment of the present application, Figure 2 ​​Fig. 1(a) is a schematic diagram of positive distortion. Positive distortion is a distortion phenomenon that the edge region of an imaging picture is stretched radially relative to the center region of the imaging picture due to factors such as lens design and light path refraction characteristics of an optical system, so that the actual imaging deviates from the ideal perspective projection model. Figure 2 Fig. 1(b) is a schematic diagram of negative distortion. Negative distortion is a distortion phenomenon that the edge region of an imaging picture is compressed radially relative to the center region of the imaging picture due to factors such as light path design and lens curvature / refractive index distribution of an optical system, so that the actual imaging deviates from the ideal perspective projection model. The gray fill represents the size of the object itself, and the area contained by the gray line is the size of the imaging. For example, the camera used is a wide-angle fisheye lens, the imaging direction is from top to bottom, and the distortion is positive barrel distortion, as shown in Figure 3 and Figure 4 . Figure 3 Fig. 2 is a physical effect diagram of the corner lens imaging from top to bottom according to an embodiment of the present application, Figure 4 Fig. 3 is a schematic diagram of positive barrel distortion according to an embodiment of the present application, as shown in Figure 3 and Figure 4 It can be seen that the physical effect diagram of the wide-angle lens imaging from top to bottom corresponds to positive barrel distortion, so in order to ensure the accuracy of detection, the imaging picture needs to be corrected. The distortion characteristic of the wide-angle lens is that the farther away from the center of the picture, the greater the distortion, and there is no fixed distortion parameter, so the distortion in different regions of the lens needs to be corrected, and an algorithm is added to eliminate the distortion. Distortion is affected by the following factors: the distance of the object, the angle of view of imaging, the closer to the center of the lens, the smaller the distortion.

[0037] In the prior art, the main algorithm for eliminating distortion is described as follows:

[0038] The polynomial distortion model is based on the relationship between the incident angle θ and the distance r of the imaging point to the optical center to describe the distortion. The formula is:

[0039]

[0040] wherein, is the distance of the imaging point to the optical center after distortion, is the angle between the incident light and the optical axis, is the focal length of the camera, , , , , is the distortion coefficient, which is usually taken as 1 to simplify the calculation. The specific correction method is shown in , Figure 5 , Figure 6 , Figure 7 , Figure 5A geometric diagram of a pinhole camera imaging model provided by an embodiment of the present application, Figure 6 A projection curve diagram of different distortion models provided by an embodiment of the present application, Figure 7 A geometric projection model diagram of a fisheye lens provided by an embodiment of the present application, visual monitoring, device positioning, and safety area identification in an industrial scene all rely on such a basic model.

[0041] However, this method needs to obtain the incident angle in the corresponding three-dimensional space, and is suitable for image correction after imaging. In the safety protection process, a monocular camera cannot measure the incident angle in real time, and if a binocular is added to calculate the incident angle, the calculation process is too slow and is not suitable for actual real-time determination business scenarios. Therefore, there are certain safety problems in safety protection. The present application is proposed based on the above problems, and the present application corrects the coordinates by matching the lateral distortion rate and the longitudinal distortion rate, solves the problem that the image distortion leads to inaccurate image-based coordinate positioning and safety protection, greatly improves the accuracy of the position judgment of the moving object, and ensures the effectiveness of the safety protection measures.

[0042] The safety protection method according to the embodiments of the present application is described below with reference to the accompanying drawings.

[0043] Figure 8 A flowchart of the safety protection method of an embodiment of the present application.

[0044] For example, as shown in Figure 8 The safety protection method includes the following steps:

[0045] In step S801, an image of a moving object in a to-be-detected region is obtained.

[0046] The to-be-detected region refers to a specific spatial range that needs to be observed, identified, or protected in real time by technical means; and the moving object refers to an object that changes position in the to-be-detected region.

[0047] Specifically, the image of the moving object in the to-be-detected region can be directly captured by continuously shooting or recording real-time video by installing an industrial camera, a high-definition camera, or a fisheye lens on the boundary of the region, or by combining an infrared sensor or a microwave radar and other auxiliary devices to obtain the image of the moving object.

[0048] In step S802, the horizontal coordinate, the vertical coordinate, and the measurement angle of the moving object are determined according to the image of the moving object, and the lateral distortion rate and the longitudinal distortion rate of the measurement angle are matched from a preset mapping relationship.

[0049] Wherein, the abscissa refers to a value used to represent the horizontal position of an object in a two-dimensional coordinate system. The ordinate refers to a value used to represent the vertical position of an object in a two-dimensional coordinate system; the measured angle refers to the azimuth of the moving object relative to the preset reference point, that is, the included angle between the line connecting the object and the reference point and the X axis; the preset mapping relationship can be a mapping relationship preset by the user, can be a mapping relationship obtained through a limited number of experiments, or can be a mapping relationship obtained through a limited number of computer simulations. The construction process of the preset mapping relationship will be described in detail later, and will not be limited here. The lateral distortion rate is the degree of deviation of the optical distortion of the lens in the horizontal direction (X axis), which is usually expressed as a proportion or coefficient. The longitudinal distortion rate is the degree of deviation of the optical distortion of the lens in the vertical direction (Y axis), which is also expressed as a proportion or coefficient.

[0050] Specifically, the embodiment of the present application needs to obtain the measured angle in the corresponding three-dimensional space, which is suitable for image correction after imaging. In the process of actual automation, a monocular camera cannot measure the incident angle in real time. If a binocular camera is added to calculate the measured angle, the calculation process is too slow and is not suitable for actual real-time determination business scenarios. Therefore, the distortion correction algorithm is simplified as follows: an actual standard object is imaged (such as a chessboard Figure 9 As shown in the figure, the imaging is compared with the standard object, the imaging boundary points are selected, and H, Ha, a, L, and La are measured.

[0051] The longitudinal distortion rate = Ha / (H-Ha); the lateral distortion rate = La / (L-La).

[0052] Wherein, H is the actual length of the chessboard in the vertical direction, which is the reference physical size for correction, Ha is the number of pixels occupied by the chessboard in the image in the vertical direction, which reflects the presentation effect of the actual height H after being distorted in the image, a usually refers to the vertical included angle of two adjacent sides of the chessboard, or the actual included angle of a side of the object and the horizontal line, which serves as a reference for judging the angular distortion of the image, L usually refers to the actual length of the chessboard in the horizontal direction, which is the reference physical size for horizontal correction, and La is the number of pixels occupied by the chessboard in the image in the horizontal direction, which reflects the presentation effect of the actual length L after being distorted in the image.

[0053] Specifically, as shown in the figure, Figure 10 As shown in the figure, Figure 10 The figure shows the distortion correction parameter measurement provided by an embodiment of the present application, wherein, Figure 10 In (a), the vertical distortion measurement refers to the deviation degree of the true geometric size of the object in the vertical direction and the corresponding size in the image after imaging, Figure 10The middle (b) represents the horizontal distortion measurement, which is the quantitative deviation of the real space size of the object in the horizontal direction and the corresponding horizontal size in the image after imaging. By using a standard object and distortion imaging, it is intuitively shown how to obtain the required parameters for correction by comparing the physical quantity with the pixel quantity of the imaging image.

[0054] Optionally, in some embodiments, matching the lateral distortion rate and the longitudinal distortion rate of the measurement angle from the preset mapping relationship comprises: determining whether the measurement angle is an integer angle; if the measurement angle is an integer angle, matching a target angle identical to the measurement angle from the preset mapping relationship, and taking the lateral distortion rate and the longitudinal distortion rate of the target angle as the lateral distortion rate and the longitudinal distortion rate of the measurement angle.

[0055] The integer angle refers to the value of the measurement angle being an integer. The target angle is an angle value identical to the value of the current measurement angle in the preset mapping relationship, which is a reference angle accurately matched from the mapping relationship.

[0056] Specifically, first, it is determined whether the measurement angle of the moving object obtained through image analysis is an integer.

[0057] If the measurement angle is an integer angle, a target angle identical to the integer angle is directly found in the pre-constructed mapping relationship between the angle and the distortion rate, and the lateral distortion rate and the longitudinal distortion rate corresponding to the target angle are directly used as the distortion rate of the current measurement angle. The calculation method of the measurement angle not being an integer angle will be described in detail later.

[0058] Therefore, by determining whether the measurement angle is an integer, the lateral and longitudinal distortion rates corresponding to the same integer angle are matched from the preset mapping relationship. Without additional calculation, the matching efficiency of the distortion rate is improved, and the accuracy of subsequent coordinate correction is laid a foundation.

[0059] In step S803, the corrected horizontal coordinate and the corrected vertical coordinate are determined according to the horizontal coordinate, the vertical coordinate, the lateral distortion rate and the longitudinal distortion rate of the measurement angle, and safety protection is performed according to the corrected horizontal coordinate and the corrected vertical coordinate.

[0060] The corrected horizontal coordinate is the result of distortion compensation on the original horizontal coordinate, which is obtained by calculating the original horizontal coordinate and the lateral distortion rate, and is closer to the horizontal position of the object in the actual physical space. The corrected vertical coordinate is the result of distortion compensation on the original vertical coordinate, which is obtained by calculating the original vertical coordinate and the longitudinal distortion rate, and is closer to the vertical position of the object in the actual physical space.

[0061] Specifically, the horizontal distortion rate is used to adjust the horizontal coordinate in the horizontal direction to eliminate the horizontal distortion effect of the lens at that angle; at the same time, the vertical distortion rate is used to adjust the vertical coordinate in the vertical direction to eliminate vertical distortion deviation. Based on the corrected and accurate coordinates, the position of the moving object in the actual physical space is determined, thereby triggering the corresponding safety protection action.

[0062] In some embodiments, determining the corrected abscissa and corrected ordinate based on the abscissa, ordinate, and the lateral and longitudinal distortion rates of the measurement angle includes: obtaining the corrected abscissa based on the abscissa and the lateral distortion rate of the measurement angle; and obtaining the corrected ordinate based on the ordinate and the longitudinal distortion rate of the measurement angle.

[0063] Specifically, the real-time video feed from the camera is connected to the controller of the automated equipment for signal transmission and equipment control. After image acquisition, the coordinate values ​​are determined through measurements in the vision system. , ), and the measured angle. The included angle needs to be rounded and matched to the database. Based on the rounded measured angle, the distortion rate database is called, and then the formula is used for conversion, where:

[0064]

[0065]

[0066] in, The corrected x-axis. This is the corrected ordinate. The original x-axis, The original ordinate, For the floor function, For measuring angles.

[0067] Therefore, by matching the distortion rate with integer angles and substituting it into the formula to correct the coordinates, a precise conversion from the original image coordinates to the actual physical position is achieved, providing reliable data for subsequent equipment control.

[0068] Furthermore, in some embodiments, obtaining the corrected abscissa based on the abscissa and the lateral distortion rate of the measurement angle includes: calculating a first difference between a first preset threshold and the lateral distortion rate of the measurement angle; and obtaining the corrected abscissa based on the product of the abscissa and the first difference.

[0069] The first preset threshold is a fixed value determined in advance based on the characteristics of the imaging system through experiments and calibration. It serves as the basic reference value for calculating the corrected abscissa and is used to perform calculations with the lateral distortion rate to obtain the coefficient required for correction. The first difference is the result calculated by subtracting the lateral distortion rate of the measured angle from the first preset threshold.

[0070] Specifically, a first difference value is obtained by subtracting the lateral distortion rate from the first preset threshold, and the original horizontal coordinate is multiplied by the first difference value to obtain the coordinate after eliminating the horizontal distortion.

[0071] For example, if the first preset threshold is A1 and the lateral distortion rate is B1, the first difference value is (A1-B1), and if the original horizontal coordinate is C1, the corrected horizontal coordinate is D1=C1*(A1-B1).

[0072] Further, in some embodiments, the corrected vertical coordinate is obtained according to the vertical distortion rate of the vertical coordinate and the measurement angle, including: calculating a second difference value between a second preset threshold and the vertical distortion rate of the measurement angle; and obtaining the corrected vertical coordinate according to the product of the vertical coordinate and the second difference value.

[0073] The second preset threshold is a fixed value determined in advance according to the characteristics of the imaging system through experiments, calibration, etc., as a basic reference for calculating the corrected vertical coordinate, used for operation with the vertical distortion rate to obtain the required correction coefficient. The second difference value is calculated by subtracting the vertical distortion rate of the measurement angle from the second preset threshold.

[0074] Specifically, a second difference value is obtained by subtracting the vertical distortion rate from the second preset threshold, and the original vertical coordinate is multiplied by the second difference value to obtain the coordinate after eliminating the horizontal distortion.

[0075] For example, if the second preset threshold is A2 and the vertical distortion rate is B2, the second difference value is (A2-B2), and if the original horizontal coordinate is C2, the corrected vertical coordinate is D2=C2*(A2-B2).

[0076] Therefore, by using the compensation ratio obtained by the preset threshold and the distortion rate to multiply the original coordinate, the optical distortion of the lens in the horizontal and vertical directions is respectively and specifically offset, and the accurate coordinate close to the actual physical position of the object is finally obtained, providing a reliable basis for subsequent safety protection.

[0077] Further, in some embodiments, safety protection is performed according to the corrected horizontal coordinate and the corrected vertical coordinate, including: determining a first region to which the corrected horizontal coordinate belongs and a second region to which the corrected vertical coordinate belongs; determining a target protection action according to the first region and the second region, and performing safety protection according to the target protection action.

[0078] The first area is an area divided in advance in a horizontal direction, each area corresponding to a specific risk level, such as a safe area, a warning area, and a dangerous area, for identifying the risk degree of the object in the horizontal direction. The second area is an area divided in advance in a vertical direction, consistent with the first area in logic, for identifying the risk degree of the object in the vertical direction. The target protection action is a specific protection measure matched from preset rules according to the combination of the first area and the second area, and is a targeted operation to cope with the current risk.

[0079] Specifically, the first area to which the modified horizontal coordinate belongs is determined, and the second area to which the modified vertical coordinate belongs is determined. These areas are divided in advance according to safety requirements, each area corresponding to a different risk level. The target protection action to be performed is determined from preset rules in combination with the combination of the first area and the second area, and safety protection is implemented according to the target protection action, ensuring timely response to different risk levels.

[0080] Thus, according to the modified horizontal and vertical coordinates, the first area and the second area to which the object belongs are determined, corresponding to different risk levels. In combination with the two areas, the target protection action is matched from the preset rules and executed, realizing multi-dimensional evaluation of the risk, avoiding excessive protection or insufficient protection, and effectively improving the accuracy and reliability of safety protection.

[0081] Further, in some embodiments, the target protection action is determined according to the first area and the second area, and safety protection is performed according to the target protection action, including: determining whether the first area is a preset dangerous area or the second area is a preset dangerous area; if the first area is a preset dangerous area or the second area is a preset dangerous area, stopping the equipment in the detection area immediately; otherwise, determining whether the first area is a preset warning area or the second area is a preset warning area; if the first area is a preset warning area or the second area is a preset warning area, generating a warning reminder information, and performing a warning reminder according to the warning reminder information.

[0082] The preset dangerous area can be a dangerous area preset by the user, a dangerous area obtained through a limited number of experiments, or a dangerous area obtained through a limited number of computer simulations, without specific limitation. The preset warning area can be a warning area preset by the user, a warning area obtained through a limited number of experiments, or a warning area obtained through a limited number of computer simulations, without specific limitation. The warning reminder information is a prompt content generated when the object enters the warning area. The warning reminder is a specific warning behavior performed according to the warning reminder information.

[0083] Specifically, the modified abovementioned horizontal coordinate is judged with the X coordinate interval of the three regions, and the modified vertical coordinate is judged with the Y coordinate interval of the three regions. Among them,

[0084] If , Both values are not within the scope of the warning area, and the device does not take any action.

[0085] If , Either value is within the scope of the danger area, and the device immediately stops.

[0086] If , Both values are not within the scope of the danger area, and any value is within the scope of the warning area, and the device immediately alarms.

[0087] Thus, by adopting the hierarchical protection logic of priority judgment of the danger area, the machine can be immediately stopped when the object enters the danger area, timely alarm when entering the warning area, and not interfere with the operation of the device when there is no risk, realizing full-scene safety coverage, strong safety, and flexible acquisition of the danger area and the warning area through user setting, experiment or simulation, which can adapt to various industrial environments and has good flexibility.

[0088] In summary, by obtaining the image of the moving object in the to-be-detected region, the horizontal coordinate, the vertical coordinate and the measurement angle of the moving object are determined according to the image, and the horizontal and vertical distortion rates corresponding to the measurement angle are matched from the preset mapping relationship. Finally, the corrected coordinates are determined according to the horizontal and vertical coordinates and the matched horizontal and vertical distortion rates, and the safety protection work is carried out based on the corrected coordinates. The effectiveness of safety control is ensured, and it is suitable for various scenes that need to monitor moving objects in real time and carry out safety protection.

[0089] To ensure the accuracy and pertinence of safety protection, the safety protection range of the automatic device needs to be demarcated with a risk boundary to provide clear standards for regional judgment.

[0090] Further, in some embodiments, after judging whether the first region is a preset warning region or whether the second region is a preset warning region, it further includes: if the first region is not a preset warning region and the second region is not a preset warning region, maintaining the device in the to-be-detected region.

[0091] Among them, the device in the to-be-detected region is various devices within the monitored range, and its running state is controlled by the safety protection logic.

[0092] Specifically, when it is determined that the horizontal region (the first region) where the object is located does not belong to the preset warning region, and the vertical region (the second region) also does not belong to the preset warning region (that is, the object is located in a completely safe region), no intervention is performed on the equipment in the monitoring range, and the equipment continues to work according to the normal process.

[0093] Therefore, by using the determination logic that the first region and the second region are both safe regions when neither of them is a warning region, meaningless intervention on the equipment can be effectively avoided, equipment wear and tear and production interruption caused by frequent start and stop are reduced, and operation continuity and efficiency are ensured.

[0094] Further, in some embodiments, before determining whether the first region is a preset dangerous region or whether the second region is a preset dangerous region, the method further includes: obtaining a working radius of the automated equipment in the region to be detected; and determining the preset dangerous region and the preset warning region according to the working radius of the automated equipment.

[0095] The working radius is the maximum space range covered by the moving parts of the automated equipment when the automated equipment is working normally.

[0096] Specifically, the dangerous region, the warning region, and the safe region are determined by analyzing the working radius of the automated equipment. The limit positions of all moving parts are found out, as shown in FIG. 2. Figure 11 As shown in FIG. 2, the region composed of all limit points is defined as the dangerous region (X2-X1, Y2-Y1, Z2-Z1). The diagonal lines of the cross section of the entire dangerous region are found out, and the length of each diagonal line is increased by 5% in each direction. The length of the increase can also be adjusted according to the actual situation. The region defined by the increased length is the warning region, and the space beyond the region is the safe region. Usually, the region in which the diagonal lines of the cross section of the dangerous region are each increased by 10% is set as the range to be monitored.

[0097] In addition, in the case where the fisheye lens is used as the monitoring tool, the field of view and the focal length of the fisheye lens can also be selected according to the boundary values of the monitoring range. The fisheye lens is arranged directly above the equipment. First, the coordinates are calculated by using the limit values determined in the above steps, in which Z is the limit value of the Z-axis multiplied by a coefficient, and the viewing distance of the wide-angle lens is further determined. X is the limit value of the X-axis multiplied by a coefficient, and Y is the limit value of the Y-axis multiplied by a coefficient. The coefficient is recommended to be 1.2, as shown in FIG. 3. Figure 12 As shown in FIG. 3, the limit value of the Z-axis is defined as Z2-Z1, the limit value of X is defined as M2-M1, and the limit value of Y is defined as N2-N1. Based on a preset angle of view calculation formula, the maximum angle of view a is calculated.

[0098] The preset angle of view calculation formula is as follows:

[0099]

[0100] Therefore, the visible distance of the wide-angle lens is determined by the Z-value, the maximum coverage area of ​​the monitoring is determined by the X and Y values, and the maximum viewing angle is calculated based on a preset viewing angle calculation formula. After obtaining the maximum viewing angle, this embodiment of the application can finally determine the lens selection based on the maximum viewing angle. It should be noted that the method of finally determining the lens selection based on the maximum viewing angle can adopt the methods in related technologies, which will not be described in detail here to avoid redundancy.

[0101] The process of constructing the preset mapping relationship will be explained in detail below.

[0102] Optionally, in some embodiments, before matching the lateral and longitudinal distortion rates of the measured angles from a preset mapping relationship, the method further includes: acquiring an image of the region to be detected; establishing a two-dimensional rectangular coordinate system with the geometric center point of the image of the region to be detected as the origin, and dividing the image of the region to be detected into multiple angles according to a preset direction, wherein the horizontal axis of the two-dimensional rectangular coordinate system is aligned with the horizontal direction of the image of the region to be detected, and the vertical axis of the two-dimensional rectangular coordinate system is aligned with the vertical direction of the image of the region to be detected; calculating the lateral and longitudinal distortion rates of at least some angles, and constructing a preset mapping relationship based on the lateral and longitudinal distortion rates of at least some angles.

[0103] Here, the geometric center point is the geometrically central location of the image region to be detected. The preset direction can be a direction pre-defined by the user, a direction obtained through a finite number of experiments, or a direction obtained through a finite number of computer simulations; no specific limitations are imposed here.

[0104] Specifically, firstly, a complete image of the area to be detected is acquired, serving as the basis for establishing the coordinate system and dividing angles. A two-dimensional Cartesian coordinate system is established with the geometric center of the image as the origin: the horizontal axis (X-axis) is aligned with the horizontal direction of the image, and the vertical axis (Y-axis) is aligned with the vertical direction of the image. Following a preset direction, the entire image plane is divided into multiple continuous angular intervals, covering all directions of the image. For at least some of the divided angles, the lateral and longitudinal distortion rates are calculated. The calculation method typically involves comparing the actual size of a standard object at the corresponding angle with the image size. The angle is divided into 360 equal parts, measured, and a corresponding model is established. The angle α, lateral distortion rate, and longitudinal distortion rate data are bound together, a database is established, and the data is entered into the system, forming a preset mapping relationship for rapid matching and retrieval during subsequent actual measurements.

[0105] Therefore, through the coordinate system establishment, angle division and distortion rate calculation, the mapping relationship constructed is accurate and comprehensive, which provides a reliable basis for fast matching of distortion rate in subsequent actual measurement, greatly improves the distortion rate calling efficiency, and lays a solid foundation for the accuracy of coordinate correction, and further ensures the accuracy of safety protection.

[0106] The calculation method of the measurement angle which is not an integer angle will be described in detail below.

[0107] Optionally, in some embodiments, after judging whether the measurement angle is an integer angle, the method further includes: if the measurement angle is not an integer angle, performing an integer operation on the measurement angle to obtain an integer angle; and matching a target angle identical to the integer angle from the preset mapping relationship, and taking the lateral distortion rate and the longitudinal distortion rate of the target angle as the lateral distortion rate and the longitudinal distortion rate of the measurement angle.

[0108] wherein the non-integer angle refers to a case where the value of the measurement angle is a decimal (non-integer).

[0109] Specifically, it is judged that the measurement angle of the moving object is not an integer, and an integer angle is obtained by performing an integer operation on the angle; and a target angle identical to the integer angle is found from the preset angle and distortion rate mapping relationship, and the lateral distortion rate and the longitudinal distortion rate corresponding to the target angle are directly used as the distortion rate of the current non-integer measurement angle.

[0110] Therefore, the distortion rate matching process of the non-integer angle is simplified by the integer operation, ensuring that the preset distortion rate data can still be quickly called when the angle is not an integer.

[0111] Further, in order to avoid the mechanism that the automatic equipment is mistakenly triggered to stop urgently due to its own movement, the embodiments of the present application can divide the to-be-detected region.

[0112] As a possible implementation manner, in some embodiments, the to-be-detected region includes a shielding region and a detectable region, wherein the moving object belongs to the detectable region.

[0113] wherein the shielding region is a part of the to-be-detected region which is shielded, and the detectable region is a part of the to-be-detected region other than the shielding region, which is an effective region for actually detecting, judging the position and state of the moving object, and the like.

[0114] Specifically, by measuring and calculating the distance between the lens picture and the picture center, the threshold of the signal trigger is set, and the range of the control area in the lens picture is set. Because the automatic equipment has relative motion when running, and the motion unit is in the dangerous area, the object that needs to be shielded needs to be set, otherwise the equipment will trigger the signal under the motion condition, resulting in emergency stop. The function development is added on the visual monitoring interface, and the picture area is set → the mask area is set to shield the area of the moving part. As shown in Figure 13 Figure 14 The circuit board area division schematic diagram provided by an embodiment of the present application includes a check area 1301 and a shielding area 1302. By dividing the detection area and the shielding area, the problem area can be accurately focused, and the detection efficiency is improved.

[0115] Therefore, by analyzing the limit position of the moving part of the automatic equipment to determine the dangerous area, the warning area and the safe area are expanded based on this, and a suitable fisheye lens is selected according to the monitoring range parameters; at the same time, the to-be-detected area is divided into a shielding area and a detectable area, and accurate detection is realized by setting the mask area, which not only ensures that the area division is in line with the characteristics of the equipment, but also improves the detection efficiency and accuracy.

[0116] According to the safety protection method provided by the embodiment of the present application, the horizontal and vertical coordinates of the moving object and the measured angle are determined, the horizontal distortion rate and the vertical distortion rate are matched from the preset mapping relationship, and the distorted horizontal and vertical coordinates are corrected, thereby solving the problem that the image of the moving object is distorted when the fisheye lens is imaged, resulting in inaccurate coordinate positioning and safety protection based on the image. The accuracy of the position judgment of the moving object is greatly improved, and the effectiveness of the safety protection measure is guaranteed.

[0117] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment.

[0118] Secondly, the safety protection device according to the embodiment of the present application is described with reference to the accompanying drawings.

[0119] Figure 14 is a block schematic diagram of the safety protection device of an embodiment of the present application.

[0120] As Figure 15 shown, the safety protection device 10 includes an acquisition module 100, a matching module 200 and a correction module 300.

[0121] ​The acquisition module 100 is configured to acquire an image of a moving object in a detection area; the matching module 200 is configured to determine a horizontal coordinate, a vertical coordinate and a measuring angle of the moving object according to the image of the moving object, and match a horizontal distortion rate and a vertical distortion rate of the measuring angle from a preset mapping relationship; and the correction module 300 is configured to determine a corrected horizontal coordinate and a corrected vertical coordinate according to the horizontal coordinate, the vertical coordinate, the horizontal distortion rate and the vertical distortion rate of the measuring angle, and perform safety protection according to the corrected horizontal coordinate and the corrected vertical coordinate.

[0122] Further, in some embodiments, before matching the horizontal distortion rate and the vertical distortion rate of the measuring angle from the preset mapping relationship, the matching module 200 is further configured to: acquire an image of the detection area; establish a two-dimensional rectangular coordinate system with a geometric center point of the image of the detection area as an origin, and divide the image of the detection area into a plurality of angles according to a preset direction, wherein a horizontal axis of the two-dimensional rectangular coordinate system is aligned with a horizontal direction of the image of the detection area, and a vertical axis of the two-dimensional rectangular coordinate system is aligned with a vertical direction of the image of the detection area; calculate the horizontal distortion rate and the vertical distortion rate of at least part of the angles, and construct the preset mapping relationship according to the horizontal distortion rate and the vertical distortion rate of at least part of the angles.

[0123] Further, in some embodiments, the matching module 200 is further configured to: determine whether the measuring angle is an integer angle; if the measuring angle is an integer angle, match a target angle identical to the measuring angle from the preset mapping relationship, and take the horizontal distortion rate and the vertical distortion rate of the target angle as the horizontal distortion rate and the vertical distortion rate of the measuring angle.

[0124] Further, in some embodiments, after determining whether the measuring angle is an integer angle, the correction module 300 is further configured to: if the measuring angle is not an integer angle, perform an integer operation on the measuring angle to obtain an integer angle; match a target angle identical to the integer angle from the preset mapping relationship, and take the horizontal distortion rate and the vertical distortion rate of the target angle as the horizontal distortion rate and the vertical distortion rate of the measuring angle.

[0125] Further, in some embodiments, the correction module 300 is further configured to: obtain the corrected horizontal coordinate according to the horizontal coordinate and the horizontal distortion rate of the measuring angle; and obtain the corrected vertical coordinate according to the vertical coordinate and the vertical distortion rate of the measuring angle.

[0126] Further, in some embodiments, the correction module 300 is further configured to: calculate a first difference value between a first preset threshold and the horizontal distortion rate of the measuring angle; and obtain the corrected horizontal coordinate according to a product of the horizontal coordinate and the first difference value.

[0127] Further, in some embodiments, the correction module 300 is further configured to: calculate a second difference between the second preset threshold and the longitudinal distortion rate of the measurement angle; and obtain the corrected longitudinal coordinate according to the product of the longitudinal coordinate and the second difference.

[0128] Further, in some embodiments, the correction module 300 is further configured to: determine a first region to which the corrected longitudinal coordinate belongs and a second region to which the corrected longitudinal coordinate belongs; determine a target protection action according to the first region and the second region; and perform safety protection according to the target protection action.

[0129] Further, in some embodiments, determining the target protection action according to the first region and the second region and performing safety protection according to the target protection action comprises: determining whether the first region is a preset dangerous region or whether the second region is a preset dangerous region; if the first region is the preset dangerous region or the second region is the preset dangerous region, immediately stopping the equipment in the detection area from working; otherwise, determining whether the first region is a preset warning region or whether the second region is a preset warning region; if the first region is the preset warning region or the second region is the preset warning region, generating a warning prompt information and performing a warning prompt according to the warning prompt information.

[0130] Further, in some embodiments, after determining whether the first region is the preset warning region or whether the second region is the preset warning region, the method further comprises: if the first region is not the preset warning region and the second region is not the preset warning region, maintaining the equipment in the detection area working.

[0131] Further, in some embodiments, before determining whether the first region is the preset dangerous region or whether the second region is the preset dangerous region, the method further comprises: obtaining a working radius of the automated equipment in the detection area; and determining the preset dangerous region and the preset warning region according to the working radius of the automated equipment.

[0132] Further, in some embodiments, the detection area comprises a shielding area and a detectable area, wherein the moving object belongs to the detectable area.

[0133] The features of the embodiments of the safety protection device can be referred to the related descriptions of the embodiments of the safety protection method, which will not be repeated here.

[0134] According to the safety protection device provided by the embodiment of the application, the transverse and longitudinal coordinates and the measurement angle of the moving object are determined, the transverse distortion rate and the longitudinal distortion rate are matched from the preset mapping relationship, the distorted transverse and longitudinal coordinates are corrected, the problem that the image of the moving object is distorted when the fisheye lens is imaged, and the coordinate positioning and safety protection based on the image are inaccurate is solved, the accuracy of the position judgment of the moving object is greatly improved, and the effectiveness of the safety protection measure is ensured.

[0135] Figure 15 The structural schematic diagram of the electronic device provided by the embodiment of the application is provided. The electronic device can include:

[0136] The memory 1501, the processor 1502 and the computer program stored in the memory 1501 and executable on the processor 1502.

[0137] The processor 1502 implements the safety protection method provided in the above embodiment when executing the program.

[0138] Further, the electronic device further includes:

[0139] The communication interface 1503 is used for communication between the memory 1501 and the processor 1502.

[0140] The memory 1501 is used for storing the computer program executable on the processor 1502.

[0141] The memory 1501 can include a high-speed RAM (Random Access Memory, random access memory) memory, and can also include a non-volatile memory, for example, at least one disk memory.

[0142] If the memory 1501, the processor 1502 and the communication interface 1503 are independently implemented, the communication interface 1503, the memory 1501 and the processor 1502 can be connected to each other through a bus and complete the communication between each other. The bus can be an ISA (Industry Standard Architecture, industry standard architecture) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnect) bus or an EISA (Extended Industry Standard Architecture, extended industry standard architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, ​ In the figure, only one thick line is used to represent, but it does not mean that there is only one bus or one type of bus.

[0143] Optionally, in a specific implementation, if the memory 1501, the processor 1502 and the communication interface 1503 are integrated on a chip, the memory 1501, the processor 1502 and the communication interface 1503 can complete the communication among each other through an internal interface.

[0144] The processor 1502 can be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or an integrated circuit configured to implement one or more embodiments of the present application. The embodiments of the present application also provide a non-volatile computer readable storage medium, which stores a computer program, wherein the computer program is configured to perform the steps in any of the above security protection method embodiments when executed.

[0145] In an example embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0146] The embodiments of the present application also provide a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the steps in any of the above security protection method embodiments.

[0147] The embodiments of the present application also provide another computer program product, which includes a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in any of the above security protection method embodiments. The skilled person can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0148] The above specifically describes the safety protection method provided by the present application. The principle and implementation manner of the present application are described by using specific examples, and the above description of the embodiments is only used to help understand the method of the present application and the core idea thereof. It should be pointed out that, for those skilled in the art, some improvements and modifications can be made to the present application without departing from the principle of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A security protection method, characterized in that, Includes the following steps: Acquire images of moving objects within the detection area; The horizontal coordinate, vertical coordinate, and measurement angle of the moving object are determined based on the image of the moving object, and the horizontal distortion rate and vertical distortion rate of the measurement angle are matched from a preset mapping relationship. The corrected abscissa and ordinate are determined based on the abscissa, the ordinate, the lateral distortion rate and the longitudinal distortion rate of the measured angle, and safety protection is performed based on the corrected abscissa and ordinate. The measured angle refers to the azimuth angle of the moving object relative to a preset reference point, that is, the angle between the line connecting the object and the reference point and the X-axis.

2. The method according to claim 1, characterized in that, Before matching the lateral and longitudinal distortion rates of the measured angle from the preset mapping relationship, the method further includes: Acquire an image of the region to be detected; A two-dimensional rectangular coordinate system is established with the geometric center point of the image of the region to be detected as the origin, and the image of the region to be detected is divided into multiple angles according to a preset direction. The horizontal axis of the two-dimensional rectangular coordinate system is aligned with the horizontal direction of the image of the region to be detected, and the vertical axis of the two-dimensional rectangular coordinate system is aligned with the vertical direction of the image of the region to be detected. Calculate the lateral and longitudinal distortion rates for at least a portion of the angles, and construct the preset mapping relationship based on the lateral and longitudinal distortion rates for at least a portion of the angles.

3. The method according to claim 1 or 2, characterized in that, The step of matching the lateral and longitudinal distortion rates of the measured angle from a preset mapping relationship includes: Determine whether the measured angle is an integer angle; If the measured angle is an integer angle, then a target angle with the same value as the measured angle is matched from the preset mapping relationship, and the lateral distortion rate and longitudinal distortion rate of the target angle are used as the lateral distortion rate and longitudinal distortion rate of the measured angle.

4. The method according to claim 3, characterized in that, After determining whether the measured angle is an integer angle, the method further includes: If the measured angle is not an integer angle, then the measured angle is rounded down to obtain the rounded angle. Match the target angle that is the same as the rounded angle from the preset mapping relationship, and use the lateral distortion rate and longitudinal distortion rate of the target angle as the lateral distortion rate and longitudinal distortion rate of the measured angle.

5. The method according to claim 1, characterized in that, The step of determining the corrected abscissa and corrected ordinate based on the abscissa, the ordinate, the lateral distortion rate of the measured angle, and the longitudinal distortion rate includes: The corrected abscissa is obtained based on the abscissa and the lateral distortion rate of the measured angle; The corrected ordinate is obtained based on the longitudinal distortion rate of the ordinate and the measured angle.

6. The method according to claim 5, characterized in that, The step of obtaining the corrected abscissa based on the abscissa and the lateral distortion rate of the measured angle includes: Calculate the first difference between the first preset threshold and the lateral distortion rate of the measured angle; The corrected horizontal coordinate is obtained by multiplying the horizontal coordinate and the first difference.

7. The method according to claim 5, characterized in that, The step of obtaining the corrected ordinate based on the longitudinal distortion rate of the ordinate and the measured angle includes: Calculate the second difference between the second preset threshold and the longitudinal distortion rate of the measured angle; The corrected ordinate is obtained by multiplying the ordinate and the second difference.

8. The method according to claim 1, characterized in that, The security protection based on the corrected horizontal and vertical coordinates includes: Determine the first region to which the corrected horizontal coordinate belongs and the second region to which the corrected vertical coordinate belongs; The target protection action is determined based on the first area and the second area, and security protection is performed based on the target protection action.

9. The method according to claim 8, characterized in that, The step of determining the target protection action based on the first region and the second region, and performing security protection based on the target protection action, includes: Determine whether the first area is a preset danger zone, or whether the second area is the preset danger zone; If the first area is the preset danger area, or the second area is the preset danger area, then the equipment in the area to be detected is stopped immediately; otherwise, it is determined whether the first area is the preset warning area, or whether the second area is the preset warning area. If the first area is the preset warning area, or the second area is the preset warning area, then a warning reminder message is generated, and a warning reminder is given based on the warning reminder message.

10. The method according to claim 9, characterized in that, After determining whether the first area is a preset warning area, or whether the second area is a preset warning area, the method further includes: If the first area is not a preset warning area and the second area is not a preset warning area, then the device in the area to be detected will continue to operate.

11. The method according to claim 9, characterized in that, Before determining whether the first area is a preset danger zone, or whether the second area is the preset danger zone, the method further includes: Obtain the operating radius of the automated equipment within the area to be detected; The preset danger zone and the preset warning zone are determined based on the operating radius of the automated equipment.

12. The method according to claim 1, characterized in that, The area to be detected includes a shielded area and a detectable area, wherein the moving object belongs to the detectable area.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for implementing the steps of the security protection method as described in any one of claims 1 to 12 when executing the computer program.

14. A non-volatile computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the security protection method as described in any one of claims 1 to 12.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the security protection method as described in any one of claims 1 to 12.

Images