Security protection method, electronic equipment, storage medium and computer program

Abstract

The invention discloses a safety protection method, electronic equipment, a storage medium and a computer program, and relates to the technical field of machine vision, and the method comprises the steps: obtaining an image of a moving object in a to-be-detected region; determining an abscissa, an ordinate and a measurement angle of the moving object according to the image of the moving object, and matching a transverse distortion rate and a longitudinal distortion rate of the measurement angle from a preset mapping relation; and determining a corrected horizontal coordinate and a corrected vertical coordinate according to the horizontal coordinate, the vertical coordinate, and the horizontal distortion rate and the vertical distortion rate of the measurement angle, and carrying out safety protection according to the corrected horizontal coordinate and the corrected vertical coordinate. Therefore, through distortion rate matching and coordinate correction, the problem of inaccurate image-based coordinate positioning and safety protection caused by distortion of the moving object image during fisheye lens imaging is solved, the accuracy of position judgment of the moving object is greatly improved, and the effectiveness of safety protection measures is guaranteed.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a safety protection method, electronic equipment, storage medium and computer program. Background Art

[0002] With the deepening of industrial automation, the widespread use of automated equipment in production lines has greatly improved production efficiency. However, the high-speed operation of equipment poses a potential risk of collision with personnel and foreign objects, making real-time monitoring and safety protection of the area to be inspected a key link in industrial safety.

[0003] Currently, the industry mostly uses industrial lenses or fisheye lenses to capture images, detect moving objects and extract coordinates through machine vision algorithms, and trigger protective actions based on preset areas. Some solutions will set shielding areas for the equipment's own moving parts to reduce misjudgments.

[0004] However, the distortion problem of the fisheye lens leads to insufficient positioning accuracy of moving objects, and there is a lack of standardized distortion correction logic. The division of dangerous areas and warning areas is often out of line with the actual operating radius of the equipment. The protective action triggering logic is also not standardized, which can easily lead to false triggering or missed detection, and needs to be solved urgently. Summary of the Invention

[0005] The present application provides a safety protection method to at least solve the problem in related technologies that the image of a moving object is distorted when imaging with a fisheye lens, resulting in inaccurate image-based coordinate positioning and safety protection. It greatly improves the accuracy of judging the position of the moving object and ensures the effectiveness of safety protection measures.

[0006] To achieve the above objectives, the first embodiment of the present application provides a security protection method, which includes the following steps: Acquire an image of a moving object in the area to be detected; determining the horizontal coordinate, vertical coordinate, and measurement angle of the moving object according to the image of the moving object, and matching the horizontal distortion rate and the vertical distortion rate of the measurement angle from a preset mapping relationship; A corrected abscissa and a corrected ordinate are determined according to the abscissa, the ordinate, the lateral distortion rate, and the longitudinal distortion rate of the measured angle, and safety protection is performed according to the corrected abscissa and the corrected ordinate.

[0007] According to the safety protection method proposed in the embodiment of the present application, by determining the horizontal and vertical coordinates and measurement angle of the moving object, the horizontal distortion rate and the vertical distortion rate are matched from a preset mapping relationship, thereby correcting the distorted horizontal and vertical coordinates. This solves the problem of inaccurate image-based coordinate positioning and safety protection caused by distortion of the image of the moving object when imaging with a fisheye lens, greatly improving the accuracy of judging the position of the moving object and ensuring the effectiveness of safety protection measures.

[0008] To achieve the above-mentioned purpose, the second embodiment of the present application further provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned security protection methods when executing the computer program.

[0009] To achieve the above-mentioned purpose, the third aspect embodiment of the present application also provides a non-volatile computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned security protection methods are implemented.

[0010] To achieve the above-mentioned objectives, the fourth aspect of the present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned security protection methods when executed by a processor.

[0011] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 A schematic diagram of pincushion distortion and barrel distortion provided in one embodiment of the present application; Figure 2 A schematic diagram of positive distortion and negative distortion provided in one embodiment of the present application; Figure 3 This is a top-down imaging effect diagram of a corner lens provided in one embodiment of the present application; Figure 4 A schematic diagram of forward barrel distortion provided by one embodiment of the present application; Figure 5 A geometric diagram of a pinhole camera imaging model provided in one embodiment of the present application; Figure 6A schematic diagram of projection curves of different distortion models provided in one embodiment of the present application; Figure 7 A schematic diagram of a geometric projection model of a fisheye lens provided in one embodiment of the present application; Figure 8 A flow chart of the security protection method provided in the embodiment of the present application; Figure 9 A schematic diagram of a checkerboard provided for one embodiment of the present application; Figure 10 A schematic diagram of distortion correction parameter measurement provided in one embodiment of the present application; Figure 11 A schematic diagram of a layered safety zone from a camera perspective provided in one embodiment of the present application; Figure 12 A schematic diagram of nested danger-warning-safety zones in the camera field of view provided in one embodiment of the present application; Figure 13 A schematic diagram of circuit board area division provided in one embodiment of the present application; Figure 14 A schematic block diagram of a safety protection device provided according to an embodiment of the present application; Figure 15 A schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0015] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0016] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0017] Before introducing the safety protection method of the embodiment of the present application, a brief introduction to the distortion mode of the current wide-angle fisheye lens is given.

[0018] The wide-angle fisheye lens can detect a large coverage area, but because of its short focal length, the closer the image is to the edge, the greater the distortion. Distortion is the phenomenon that the size ratio of the image formed by the lens deviates from the actual object. There are usually two types of distortion: pincushion distortion and barrel distortion. Figure 1 As shown, Figure 1 A schematic diagram of pincushion distortion and barrel distortion provided in one embodiment of the present application is shown. Figure 1 (a) in the middle is a schematic diagram of pincushion distortion. Pincushion distortion is a phenomenon in which the true straight lines of an object bend and shrink toward the center of the image after imaging. The overall shape resembles the surface contour of a pillow, causing the size of objects at the edge of the image to be compressed and out of proportion with the central area. Figure 1 Figure (b) illustrates barrel distortion. Barrel distortion occurs when a straight line on an object bends and expands outward from the image after being formed. This distortion resembles the side profile of a barrel, stretching objects at the edges of the image and significantly out of proportion with the center. These two phenomena are caused by different magnifications in the X and Y directions after light passes through the lens at different locations.

[0019] Distortion can be divided into two categories according to other classifications, namely positive distortion and negative distortion. Positive distortion and negative distortion are defined according to the size of the image and the actual object. If the image size is enlarged compared to the original size, it is defined as positive distortion. If the image size is smaller than the original size, it is defined as negative distortion. Figure 2 As shown, Figure 2 A schematic diagram of positive distortion and negative distortion provided in one embodiment of the present application is shown. Figure 2 (a) is a schematic diagram of positive distortion. Positive distortion is a distortion phenomenon in which objects in the edge areas of the image are radially stretched relative to objects in the center area due to factors such as lens design and the refractive properties of the optical path. This causes the actual image to deviate from the ideal perspective projection model. Figure 2 (b) is a schematic diagram of negative distortion. Negative distortion is a distortion phenomenon in which objects in the edge area of ​​the image are radially compressed relative to objects in the center area due to factors such as the optical path design and lens curvature / refractive index distribution of the optical system, which ultimately causes the actual image to deviate from the ideal perspective projection model. The gray fill represents the size of the object itself, and the area enclosed by the gray lines is the size of the image. For example, the camera used is a wide-angle fisheye lens, and the imaging direction is from top to bottom, so the distortion presented is positive barrel distortion, such as Figure 3 and Figure 4 As shown, Figure 3 This is a top-down image of an angle lens provided in one embodiment of the present application. Figure 4 A schematic diagram of forward barrel distortion provided by an embodiment of the present application is shown in FIG. Figure 3 and Figure 4It can be seen that the top-down image of a wide-angle lens exhibits positive barrel distortion. Therefore, to ensure accurate detection, the captured image must be corrected. Wide-angle lens distortion increases with distance from the image center, and there is no fixed distortion parameter. Therefore, distortion correction is required for different areas of the lens, and algorithms are implemented to eliminate distortion. Distortion is affected by several factors, including object distance and the viewing angle. Distortion decreases with distance to the center of the lens.

[0020] In the prior art, the main algorithms for eliminating distortion are described as follows: The polynomial distortion model describes the distortion based on the relationship between the incident angle θ and the distance r from the imaging point to the optical center. Its formula is:

[0021] in, is the distance from the distorted imaging point to the optical center, is the angle between the incident ray and the optical axis, is the focal length of the camera, , , , , is the distortion coefficient, usually is taken as 1 to simplify the calculation. The specific correction method is as follows Figure 5 、 Figure 6 、 Figure 7 As shown, Figure 5 A geometric diagram of a pinhole camera imaging model provided in one embodiment of the present application, Figure 6 This is a schematic diagram of projection curves of different distortion models provided in one embodiment of the present application. Figure 7 A schematic diagram of the geometric projection model of a fisheye lens provided for one embodiment of the present application. Visual monitoring, equipment positioning, and safe area identification in industrial scenarios all rely on this type of basic model.

[0022] However, this method requires obtaining the corresponding incident angle in the three-dimensional space, which is suitable for image correction after imaging. During the security protection process, the monocular camera cannot measure the incident angle in real time. If binoculars are added to calculate the incident angle, the calculation process is too slow and is not suitable for actual real-time judgment business scenarios. Therefore, there are certain security issues in security protection. This application proposes a security protection method based on the above problems. By matching the lateral distortion rate and the longitudinal distortion rate for coordinate correction, it solves the problem of inaccurate image-based coordinate positioning and security protection due to image distortion, greatly improves the accuracy of position judgment of moving objects, and ensures the effectiveness of security protection measures.

[0023] The following describes the security protection method proposed according to the embodiments of the present application with reference to the accompanying drawings.

[0024] Figure 8 This is a flowchart of a security protection method according to an embodiment of the present application.

[0025] For example, Figure 8 As shown, the security protection method includes the following steps: In step S801, an image of a moving object in a region to be detected is acquired.

[0026] Among them, the area to be detected refers to a specific spatial range that needs to be observed, identified or protected in real time through technical means; a moving object refers to an object whose position changes within the area to be detected.

[0027] Specifically, images of moving objects in the area to be detected can be obtained by installing industrial cameras, high-definition cameras or fisheye lenses at the boundary of the area, and directly capturing dynamic scenes in the area through continuous shooting or real-time recording. Images of moving objects can also be obtained by combining auxiliary equipment such as infrared sensors or microwave radars.

[0028] In step S802 , the horizontal coordinate, vertical coordinate and measurement angle of the moving object are determined according to the image of the moving object, and the horizontal distortion rate and the vertical distortion rate of the measurement angle are matched from a preset mapping relationship.

[0029] Among them, the horizontal coordinate refers to the numerical value used to represent the horizontal position of an object in a two-dimensional coordinate system. The vertical coordinate refers to the numerical value used to represent the vertical position of an object in a two-dimensional coordinate system; the measurement angle refers to the azimuth of the moving object relative to a preset reference point, that is, the angle between the line connecting the object and the reference point and the X-axis; the preset mapping relationship can be a mapping relationship pre-set by the user, a mapping relationship obtained through a limited number of experiments, or a mapping relationship obtained through a limited number of computer simulations. The embodiment of this application will explain the construction process of the preset mapping relationship in detail later, and no specific limitation is made here. The lateral distortion rate is the degree of deviation of the optical distortion of the lens in the horizontal direction (X-axis), usually expressed as a ratio or coefficient. The longitudinal distortion rate is the degree of deviation of the optical distortion of the lens in the vertical direction (Y-axis), also expressed as a ratio or coefficient.

[0030] Specifically, the embodiment of the present application needs to obtain the corresponding measurement angle in the three-dimensional space, which is suitable for image correction after the image is taken. In the actual process of automated operation, the monocular camera cannot measure the incident angle in real time. If binoculars are added to calculate the measurement angle, the calculation process is too slow and is not suitable for actual real-time judgment business scenarios. Therefore, the distortion correction algorithm is simplified to take an actual standard object for imaging (such as a chessboard Figure 9 As shown in the figure), the image is compared with the standard object, the imaging boundary points are selected and H, Ha, α, L, and La are measured.

[0031] Longitudinal distortion rate = Ha / (H-Ha); lateral distortion rate = La / (L-La).

[0032] Here, H is the actual vertical length of the checkerboard and serves as the benchmark physical dimension for correction. Ha is the number of pixels occupied by the checkerboard in the vertical direction of the image, reflecting the distorted appearance of the actual height H in the image. α typically refers to the vertical angle between two adjacent sides of the checkerboard, or the actual angle between an edge of an object and the horizontal line, which serves as the benchmark for determining image angular distortion. L typically refers to the actual horizontal length of the checkerboard and serves as the benchmark physical dimension for horizontal correction. La is the number of pixels occupied by the checkerboard in the horizontal direction of the image, reflecting the distorted appearance of the actual length L in the image.

[0033] Specifically, if Figure 10 As shown, Figure 10 A schematic diagram of distortion correction parameter measurement provided by an embodiment of the present application, wherein: Figure 10 (a) in the middle represents the vertical distortion measurement, which is the degree of deviation between the actual geometric size of the object in the vertical direction and the corresponding size in the image after imaging. Figure 10 (b) shows horizontal distortion measurement, which quantifies the deviation between the object's real-world horizontal dimensions and the corresponding horizontal dimensions in the image. Using standard objects and distorted images, this method intuitively demonstrates how to obtain the necessary correction parameters by comparing physical quantities with image pixel counts.

[0034] Optionally, in some embodiments, the lateral distortion rate and longitudinal distortion rate of the measured angle are matched from a preset mapping relationship, including: determining whether the measured angle is an integer angle; if the measured angle is an integer angle, matching a target angle that is the same as the measured angle from a preset mapping relationship, and using the lateral distortion rate and longitudinal distortion rate of the target angle as the lateral distortion rate and longitudinal distortion rate of the measured angle.

[0035] Integer angles refer to angles whose values ​​are integers. Target angles are angles that match the current measured angle in a preset mapping relationship. They are reference angles that are precisely matched from the mapping relationship.

[0036] Specifically, it is first determined whether the measured angle of the moving object obtained through image analysis is an integer.

[0037] If the measured angle is an integer, the pre-established mapping between angles and distortion rates is used to find a target angle that is exactly the same as the integer. The lateral and longitudinal distortion rates corresponding to the target angle are then used as the distortion rates for the current measured angle. The calculation method for non-integer measured angles will be described in detail later in this application.

[0038] Therefore, by determining whether the measured angle is an integer, the horizontal and vertical distortion rates corresponding to the same integer angle are matched from the preset mapping relationship. This eliminates the need for additional calculations, improves the efficiency of distortion rate matching, and also lays the foundation for the accuracy of subsequent coordinate corrections.

[0039] In step S803 , a corrected abscissa and a corrected ordinate are determined according to the abscissa, the ordinate, the lateral distortion rate, and the longitudinal distortion rate of the measured angle, and safety protection is performed according to the corrected abscissa and the corrected ordinate.

[0040] The corrected horizontal coordinate is the result of distortion compensation for the original horizontal coordinate. It is calculated by adding the original horizontal coordinate and the horizontal distortion rate, and is closer to the horizontal position of the object in the actual physical space. The corrected vertical coordinate is the result of distortion compensation for the original vertical coordinate. It is calculated by adding the original vertical coordinate and the vertical distortion rate, and is closer to the vertical position of the object in the actual physical space.

[0041] Specifically, the horizontal coordinate is adjusted using the lateral distortion rate to eliminate the effect of horizontal lens distortion at that angle. Simultaneously, the vertical coordinate is adjusted using the longitudinal distortion rate to eliminate vertical distortion deviation. Based on these corrected precise coordinates, the position of the moving object in real physical space is determined, triggering the corresponding safety protection action.

[0042] In some embodiments, the corrected horizontal coordinate and the corrected vertical coordinate are determined based on the horizontal coordinate, the vertical coordinate, the horizontal distortion rate and the vertical distortion rate of the measured angle, including: obtaining the corrected horizontal coordinate based on the horizontal coordinate and the horizontal distortion rate of the measured angle; obtaining the corrected vertical coordinate based on the vertical coordinate and the vertical distortion rate of the measured angle.

[0043] Specifically, the real-time image of the lens is connected to the automation equipment controller for signal transmission and equipment control. After the image is captured, the coordinate value is determined by measurement in the visual system ( , ), and the measured angle. The angle needs to be rounded to match the database. Based on the rounded measured angle, call the distortion rate database and then perform the formula conversion, where:

[0044]

[0045] in, is the corrected horizontal coordinate, is the corrected vertical coordinate, is the original horizontal coordinate, is the original vertical coordinate, is the rounding function, To measure angles.

[0046] Therefore, by matching the distortion rate with integer angles and substituting it into the formula to correct the coordinate logic, accurate conversion from the original image coordinates to the actual physical position is achieved, providing reliable data for subsequent equipment control.

[0047] Furthermore, in some embodiments, obtaining a corrected horizontal coordinate based on the horizontal coordinate and the lateral distortion rate of the measured angle includes: calculating a first difference between a first preset threshold and the lateral distortion rate of the measured angle; and obtaining the corrected horizontal coordinate based on the product of the horizontal coordinate and the first difference.

[0048] The first preset threshold is a fixed value determined in advance based on the characteristics of the imaging system through experiments, calibration, and other methods. It serves as a basic reference for calculating the corrected horizontal coordinate and is used in conjunction with the lateral distortion rate to obtain the required correction coefficient. The first difference is calculated by subtracting the lateral distortion rate of the measured angle from the first preset threshold.

[0049] Specifically, a first difference is obtained by subtracting the lateral distortion rate from a first preset threshold, and the original horizontal coordinate is multiplied by the first difference to obtain the coordinate after eliminating the horizontal distortion.

[0050] For example, if the first preset threshold is A1 and the lateral distortion rate is B1, then the first difference is (A1-B1). If the original horizontal coordinate is C1, the corrected horizontal coordinate is D1= C1*(A1-B1).

[0051] Furthermore, in some embodiments, obtaining a corrected vertical coordinate based on the vertical coordinate and the longitudinal distortion rate of the measured angle includes: calculating a second difference between a second preset threshold and the longitudinal distortion rate of the measured angle; and obtaining the corrected vertical coordinate based on the product of the vertical coordinate and the second difference.

[0052] The second preset threshold is a fixed value determined in advance based on the characteristics of the imaging system through experiments, calibration, and other methods. It serves as a basic reference for calculating the corrected vertical coordinate and is used in conjunction with the longitudinal distortion rate to obtain the correction coefficient. The second difference is calculated by subtracting the longitudinal distortion rate of the measured angle from the second preset threshold.

[0053] Specifically, the second difference is obtained by subtracting the longitudinal distortion rate from the second preset threshold, and the original longitudinal coordinate is multiplied by the second difference to obtain the coordinate after eliminating the horizontal distortion.

[0054] For example, if the second preset threshold is A2 and the longitudinal distortion rate is B2, then the second difference is (A2-B2). If the original horizontal coordinate is C2, the corrected vertical coordinate is D2= C2*(A2-B2).

[0055] Therefore, by obtaining the compensation ratio through the preset threshold and distortion rate, and then multiplying it with the original coordinates, the optical distortion of the lens in the horizontal and vertical directions can be offset respectively, and finally accurate coordinates that are closer to the actual physical position of the object are obtained, providing a reliable basis for subsequent safety protection.

[0056] Furthermore, in some embodiments, security protection is performed based on the corrected horizontal coordinate and the corrected vertical coordinate, including: determining the first area to which the corrected horizontal coordinate belongs and the second area to which the corrected vertical coordinate belongs; determining a target protection action based on the first area and the second area, and performing security protection based on the target protection action.

[0057] The first zone is a pre-defined horizontal area, each corresponding to a specific risk level, such as a safe zone, a warning zone, or a dangerous zone, used to identify the horizontal risk level of an object. The second zone is a pre-defined vertical area, logically consistent with the first zone, used to identify the vertical risk level of an object. The targeted protection action is a specific protective measure matched from pre-defined rules based on the combination of the first and second zones. It is a targeted action to address the current risk.

[0058] Specifically, the first zone to which the action belongs is determined based on the corrected horizontal coordinate, while the second zone to which the action belongs is determined based on the corrected vertical coordinate. These zones are pre-divided based on security requirements, with each zone corresponding to a different risk level. Combining the first and second zones, the target protection action to be executed is determined from the pre-set zones and action rules. Security protection is then implemented according to the target protection action, ensuring timely response to situations at different risk levels.

[0059] Therefore, according to the corrected horizontal and vertical coordinates, the first area and the second area to which it belongs are determined respectively, corresponding to different risk levels; then combined with the two areas, the target protection actions are matched and executed from the preset rules, realizing a multi-dimensional assessment of risks, avoiding excessive or insufficient protection, and effectively improving the accuracy and reliability of safety protection.

[0060] Furthermore, in some embodiments, a target protection action is determined based on the first area and the second area, and safety protection is performed based on the target protection action, including: determining whether the first area is a preset danger area, or whether the second area is a preset danger area; if the first area is a preset danger area, or the second area is a preset danger area, then immediately stopping the operation of the equipment in the area to be detected; otherwise, determining whether the first area is a preset warning area, or whether the second area is a preset warning area; if the first area is a preset warning area, or the second area is a preset warning area, then generating warning reminder information, and performing a warning reminder based on the warning reminder information.

[0061] The preset danger zone can be a user-defined danger zone, a zone obtained through a limited number of experiments, or a zone obtained through a limited number of computer simulations, and is not specifically limited here. The preset warning zone can be a user-defined warning zone, a zone obtained through a limited number of experiments, or a zone obtained through a limited number of computer simulations, and is not specifically limited here. The warning reminder information is the prompt content generated when an object enters the warning zone. The warning reminder is the specific warning action executed based on the warning reminder information.

[0062] Specifically, the modified horizontal coordinates are compared with the X coordinate intervals of the three regions, and the modified vertical coordinates are compared with the Y coordinate intervals of the three regions. like , If both values ​​are outside the warning range, the device will not take any action.

[0063] like , If either value is within the range of the danger zone, the equipment stops immediately.

[0064] like , If both values ​​are outside the danger zone and either value is within the warning zone, the device will immediately alarm.

[0065] Therefore, by adopting a hierarchical protection logic that prioritizes dangerous areas, the machine can be shut down immediately when an object enters a dangerous area, and an alarm can be issued in time when it enters a warning area. When there is no risk, the equipment operation will not be interfered with, achieving full-scene safety coverage and strong security. Dangerous areas and warning areas can be flexibly obtained through user settings, experiments or simulations, and can adapt to a variety of industrial environments with good flexibility.

[0066] In summary, by acquiring an image of a moving object within the detection area, the object's horizontal and vertical coordinates and measurement angle are determined based on the image, and the horizontal and vertical distortion rates corresponding to the measurement angle are matched from a preset mapping relationship. Finally, the corrected coordinates are determined based on the horizontal and vertical coordinates and the matched horizontal and vertical distortion rates, and security protection is then carried out based on the corrected coordinates. This ensures the effectiveness of security control and is suitable for various scenarios requiring real-time monitoring of moving objects and security protection.

[0067] To ensure the accuracy and pertinence of security protection, the embodiment of the present application also needs to define risk boundaries for the security protection range of the automation equipment and provide clear standards for area determination.

[0068] Furthermore, in some embodiments, after determining whether the first area is a preset warning area, or whether the second area is a preset warning area, it also includes: if the first area is not a preset warning area, and the second area is not a preset warning area, then maintaining the operation of the equipment in the area to be detected.

[0069] Among them, the equipment in the area to be detected is various types of equipment within the monitored range, and its operating status is controlled by the safety protection logic.

[0070] Specifically, when it is determined that the horizontal area (first area) where the object is located does not belong to the preset warning area, and the vertical area (second area) does not belong to the preset warning area (that is, the object is in a completely safe area), no intervention will be made to the equipment within the monitoring range, and the equipment will continue to operate according to normal procedures.

[0071] Therefore, by clarifying the judgment logic that the first area and the second area are not warning areas and are therefore safe areas, meaningless intervention in the equipment can be effectively avoided, which not only reduces equipment loss and production interruption caused by frequent start and stop, but also ensures operation continuity and efficiency.

[0072] Furthermore, in some embodiments, before determining whether the first area is a preset dangerous area or whether the second area is a preset dangerous area, it also includes: obtaining the operating radius of the automation equipment in the area to be detected; and determining the preset dangerous area and the preset warning area based on the operating radius of the automation equipment.

[0073] Among them, the operating radius is the maximum spatial range that the moving parts of the automation equipment can cover when it is working normally.

[0074] Specifically, by analyzing the operating radius of the automation equipment, the danger zone, warning zone, and safety zone are determined. Find the limit positions of all moving parts, such as Figure 11As shown, the area formed by all the extreme points is defined as the danger zone (X2-X1, Y2-Y1, Z2-Z1). Find the diagonal line of the entire danger zone cross section and increase the diagonal length by 5% in each direction. The increased length can be adjusted according to the actual situation. This area is defined as the warning zone, and the space beyond this area is defined as the safe zone. Typically, the area that increases by 10% in each direction of the diagonal line of the danger zone cross section is set as the area to be monitored.

[0075] In addition, when the fisheye lens is used as a monitoring tool, the embodiment of the present application can also select the field of view and focal length of the fisheye lens according to the boundary value of the monitoring range. Place the fisheye lens directly above the device. First, calculate the coordinates using the limit values ​​determined in the above steps, where Z = the limit value of the Z axis * coefficient, and further determine the viewing distance of the wide-angle lens. X = the limit value of the X axis * coefficient, and Y = the limit value of the Y axis * coefficient. The coefficient is recommended to be 1.2, as shown in the following example. Figure 12 As shown, the limit value of the Z axis is defined as Z2-Z1, the limit value of the X axis is defined as M2-M1, and the limit value of the Y axis is defined as N2-N1. Based on the preset viewing angle calculation formula, the maximum viewing angle a is calculated. The preset viewing angle calculation formula is:

[0076] Thus, the visual distance of the wide-angle lens is determined by the Z value, the maximum coverage range of the monitoring is determined by the X and Y values, and the maximum viewing angle is calculated based on the preset viewing angle calculation formula. After obtaining the maximum viewing angle, the embodiment of the present application can ultimately determine the lens selection based on the maximum viewing angle. It should be noted that the method for ultimately determining the lens selection based on the maximum viewing angle can adopt the method in the related art, and to avoid redundancy, it will not be described in detail here.

[0077] The following is a detailed description of the process of constructing the preset mapping relationship.

[0078] Optionally, in some embodiments, before matching the lateral distortion rate and longitudinal distortion rate of the measured angle from a preset mapping relationship, it also includes: acquiring an image of the area to be detected; establishing a two-dimensional rectangular coordinate system with the geometric center point of the image of the area to be detected as the origin, and dividing the image of the area to be detected into multiple angles according to a preset direction, wherein the horizontal axis of the two-dimensional rectangular coordinate system is aligned with the horizontal direction of the image of the area to be detected, and the vertical axis of the two-dimensional rectangular coordinate system is aligned with the vertical direction of the image of the area to be detected; calculating the lateral distortion rate and longitudinal distortion rate of at least part of the angles, and constructing a preset mapping relationship based on the lateral distortion rate and longitudinal distortion rate of at least part of the angles.

[0079] The geometric center point is the geometric center point of the image of the area to be inspected. The preset direction can be a direction pre-set by the user, a direction obtained through a limited number of experiments, or a direction obtained through a limited number of computer simulations, and is not specifically limited here.

[0080] Specifically, a complete image of the area to be inspected is first captured as the basis for establishing a coordinate system and dividing angles. A two-dimensional rectangular coordinate system is established with the geometric center point of the image as the origin: the horizontal axis (X-axis) is aligned with the horizontal direction of the image, and the vertical axis (Y-axis) is aligned with the vertical direction of the image. The entire image plane is divided into multiple continuous angle intervals according to a preset direction, covering all orientations of the image. For at least some of the divided angles, the lateral and longitudinal distortion rates are calculated separately. The calculation method is usually obtained by comparing the actual size of a standard object at the corresponding angle with the image size. The angle is divided into 360 equal parts, and after measurement, a corresponding model is established. The angle α, lateral distortion rate, and longitudinal distortion rate data are bound, a database is established, and the data is entered into the system to form a preset mapping relationship for rapid matching and call during subsequent actual measurements.

[0081] Therefore, through the establishment of the coordinate system, angle division and distortion rate calculation, the mapping relationship constructed is accurate and comprehensive, providing a reliable basis for the rapid matching of distortion rates in subsequent actual measurements, greatly improving the efficiency of distortion rate calls, and laying a solid foundation for the accuracy of coordinate correction, thereby ensuring the accuracy of safety protection.

[0082] The following is a detailed description of the calculation method when the measured angle is not an integer angle.

[0083] Optionally, in some embodiments, after determining whether the measured angle is an integer angle, it also includes: if the measured angle is not an integer angle, rounding the measured angle to obtain a rounded angle; matching a target angle that is the same as the rounded angle from a preset mapping relationship, and using the lateral distortion rate and longitudinal distortion rate of the target angle as the lateral distortion rate and longitudinal distortion rate of the measured angle.

[0084] A non-integer angle refers to a situation where the value of the measured angle is a decimal (non-integer).

[0085] Specifically, it is determined that the measured angle of the moving object is not an integer, and the angle is rounded to obtain an integer angle; then, from the preset angle-distortion rate mapping relationship, a target angle that is completely consistent with the rounded angle is found, and the lateral distortion rate and longitudinal distortion rate corresponding to the target angle are directly used as the distortion rate of the current non-integer measured angle.

[0086] Therefore, the distortion rate matching process for non-integer angles is simplified by rounding, ensuring that the preset distortion rate data can still be quickly called when the angle is non-integer.

[0087] Furthermore, in order to prevent the automated equipment from mistakenly triggering an emergency stop mechanism due to its own movement, the embodiment of the present application may divide the area to be detected.

[0088] As a possible implementation manner, in some embodiments, the area to be detected includes a shielded area and a detectable area, wherein the moving object belongs to the detectable area.

[0089] The shielded area is the shielded portion of the area to be detected, and the detectable area is the portion of the area to be detected excluding the shielded area. It is the effective area for actually performing operations such as detection and determining the position and state of moving objects.

[0090] Specifically, by measuring and calculating the distance between the lens image and the center of the image, the threshold for signal triggering is set, and the range of the control area is set in the lens image. In addition, because the automation equipment has relative motion during operation, and the moving units are all in the dangerous area, it is necessary to set the objects to be shielded. Otherwise, as long as the equipment is in motion, there will be a signal trigger, resulting in an emergency stop. Add function development to the visual monitoring interface, set the image area → set the mask area, and shield the area of ​​the moving parts. Specifically, Figure 13 As shown, Figure 13 A schematic diagram of circuit board area division is provided for one embodiment of the present application, which includes a check area 1301 and a shielding area 1302. By dividing the check area and the shielding area, the problem area can be accurately focused on and the detection efficiency can be improved.

[0091] Therefore, by analyzing the extreme positions of the moving parts of the automation equipment, the dangerous area is determined, and the warning area and safe area are expanded based on this. The appropriate fisheye lens is selected according to the monitoring range parameters; at the same time, the area to be detected is divided into a shielded area and a detectable area, and accurate detection is achieved by setting the mask area, which not only ensures that the area division fits the equipment characteristics, but also improves the detection efficiency and accuracy.

[0092] According to the safety protection method proposed in the embodiment of the present application, by determining the horizontal and vertical coordinates and measurement angle of the moving object, the horizontal distortion rate and the vertical distortion rate are matched from a preset mapping relationship, thereby correcting the distorted horizontal and vertical coordinates. This solves the problem of inaccurate image-based coordinate positioning and safety protection caused by distortion of the image of the moving object when imaging with a fisheye lens, greatly improving the accuracy of judging the position of the moving object and ensuring the effectiveness of safety protection measures.

[0093] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0094] Next, the safety protection device proposed according to the embodiment of the present application is described with reference to the accompanying drawings.

[0095] Figure 14 It is a block diagram of a safety protection device according to an embodiment of the present application.

[0096] like Figure 14 As shown, the safety protection device 10 includes: an acquisition module 100, a matching module 200 and a correction module 300.

[0097] Among them, the acquisition module 100 is used to acquire an image of a moving object in the area to be detected; the matching module 200 is used to determine the horizontal coordinate, vertical coordinate and measurement angle of the moving object based on the image of the moving object, and match the horizontal distortion rate and vertical distortion rate of the measurement angle from a preset mapping relationship; the correction module 300 is used to determine the corrected horizontal coordinate and the corrected vertical coordinate based on the horizontal coordinate, vertical coordinate, and the horizontal distortion rate and vertical distortion rate of the measurement angle, and perform safety protection based on the corrected horizontal coordinate and the corrected vertical coordinate.

[0098] Furthermore, in some embodiments, before matching the lateral distortion rate and longitudinal distortion rate of the measured angle from a preset mapping relationship, the matching module 200 is also used to: obtain an image of the area to be detected; establish a two-dimensional rectangular coordinate system with the geometric center point of the image of the area to be detected as the origin, and divide the image of the area to be detected into multiple angles according to a preset direction, wherein the horizontal axis of the two-dimensional rectangular coordinate system is aligned with the horizontal direction of the image of the area to be detected, and the vertical axis of the two-dimensional rectangular coordinate system is aligned with the vertical direction of the image of the area to be detected; calculate the lateral distortion rate and longitudinal distortion rate of at least part of the angles, and construct a preset mapping relationship based on the lateral distortion rate and longitudinal distortion rate of at least part of the angles.

[0099] Furthermore, in some embodiments, the matching module 200 is also used to: determine whether the measured angle is an integer angle; if the measured angle is an integer angle, match the target angle that is the same as the measured angle from a preset mapping relationship, and use the lateral distortion rate and longitudinal distortion rate of the target angle as the lateral distortion rate and longitudinal distortion rate of the measured angle.

[0100] Furthermore, in some embodiments, after determining whether the measured angle is an integer angle, the correction module 300 is further used to: if the measured angle is not an integer angle, round the measured angle to obtain a rounded angle; match a target angle that is the same as the rounded angle from a preset mapping relationship, and use the lateral distortion rate and longitudinal distortion rate of the target angle as the lateral distortion rate and longitudinal distortion rate of the measured angle.

[0101] Furthermore, in some embodiments, the correction module 300 is further configured to: obtain a corrected horizontal coordinate according to the horizontal coordinate and the horizontal distortion rate of the measured angle; and obtain a corrected vertical coordinate according to the vertical coordinate and the vertical distortion rate of the measured angle.

[0102] Furthermore, in some embodiments, the correction module 300 is further configured to: calculate a first difference between a first preset threshold and the lateral distortion rate of the measured angle; and obtain a corrected horizontal coordinate according to the product of the horizontal coordinate and the first difference.

[0103] Furthermore, in some embodiments, the correction module 300 is further configured to: calculate a second difference between a second preset threshold and the longitudinal distortion rate of the measured angle; and obtain a corrected longitudinal coordinate according to the product of the longitudinal coordinate and the second difference.

[0104] Furthermore, in some embodiments, the correction module 300 is also used to: determine the first area to which the corrected horizontal coordinate belongs and the second area to which the corrected vertical coordinate belongs; determine the target protection action based on the first area and the second area, and perform safety protection based on the target protection action.

[0105] Furthermore, in some embodiments, a target protection action is determined based on the first area and the second area, and safety protection is performed based on the target protection action, including: determining whether the first area is a preset danger area, or whether the second area is a preset danger area; if the first area is a preset danger area, or the second area is a preset danger area, then immediately stopping the operation of the equipment in the area to be detected; otherwise, determining whether the first area is a preset warning area, or whether the second area is a preset warning area; if the first area is a preset warning area, or the second area is a preset warning area, then generating warning reminder information, and performing a warning reminder based on the warning reminder information.

[0106] Furthermore, in some embodiments, after determining whether the first area is a preset warning area, or whether the second area is a preset warning area, it also includes: if the first area is not a preset warning area, and the second area is not a preset warning area, then maintaining the operation of the equipment in the area to be detected.

[0107] Furthermore, in some embodiments, before determining whether the first area is a preset dangerous area or whether the second area is a preset dangerous area, it also includes: obtaining the operating radius of the automation equipment in the area to be detected; and determining the preset dangerous area and the preset warning area based on the operating radius of the automation equipment.

[0108] Furthermore, in some embodiments, the area to be detected includes a shielded area and a detectable area, wherein the moving object belongs to the detectable area.

[0109] For the description of the features in the embodiments corresponding to the safety protection device, please refer to the relevant description of the embodiments corresponding to the safety protection method, and no further details will be given here.

[0110] According to the safety protection device proposed in the embodiment of the present application, by determining the horizontal and vertical coordinates and measurement angle of the moving object, the horizontal distortion rate and the vertical distortion rate are matched from a preset mapping relationship, thereby correcting the distorted horizontal and vertical coordinates. This solves the problem of inaccurate image-based coordinate positioning and safety protection caused by distortion of the image of the moving object when imaging with a fisheye lens, greatly improving the accuracy of judging the position of the moving object and ensuring the effectiveness of safety protection measures.

[0111] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include: Memory 1501 , processor 1502 , and computer programs stored in the memory 1501 and executable on the processor 1502 .

[0112] When the processor 1502 executes the program, the security protection method provided in the above embodiment is implemented.

[0113] Furthermore, the electronic device further includes: The communication interface 1503 is used for communication between the memory 1501 and the processor 1502 .

[0114] The memory 1501 is used to store computer programs that can be run on the processor 1502 .

[0115] The memory 1501 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0116] If the memory 1501, processor 1502, and communication interface 1503 are implemented independently, the communication interface 1503, memory 1501, and processor 1502 can be connected to each other via a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0117] Optionally, in a specific implementation, if the memory 1501, the processor 1502 and the communication interface 1503 are integrated on a chip, the memory 1501, the processor 1502 and the communication interface 1503 can communicate with each other through an internal interface.

[0118] Processor 1502 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. The embodiments of the present application also provide a non-volatile computer-readable storage medium having a computer program stored therein, wherein the computer program is configured to execute the steps of any of the above-mentioned security protection method embodiments when executed.

[0119] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0120] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above security protection method embodiments are implemented.

[0121] The embodiments of the present application also provide another computer program product, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and the computer program, when executed by a processor, implements the steps in any of the above-mentioned security protection method embodiments. Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0122] The above is a detailed introduction to a security protection method provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core ideas. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A safety protection method, characterized in that: The following steps are involved: Acquire an image of a moving object in the area to be detected; determining the horizontal coordinate, vertical coordinate, and measurement angle of the moving object according to the image of the moving object, and matching the horizontal distortion rate and the vertical distortion rate of the measurement angle from a preset mapping relationship; A corrected abscissa and a corrected ordinate are determined according to the abscissa, the ordinate, the lateral distortion rate, and the longitudinal distortion rate of the measured angle, and safety protection is performed according to the corrected abscissa and the corrected ordinate.

2. The method according to claim 1, characterized in that Before matching the lateral distortion rate and the longitudinal distortion rate of the measurement angle from the preset mapping relationship, the method further includes: Acquiring an image of the area to be detected; Establishing a two-dimensional rectangular coordinate system with the geometric center point of the image of the area to be detected as the origin, and dividing the image of the area to be detected into multiple angles according to a preset direction, wherein the horizontal axis of the two-dimensional rectangular coordinate system is aligned with the horizontal direction of the image of the area to be detected, and the vertical axis of the two-dimensional rectangular coordinate system is aligned with the vertical direction of the image of the area to be detected; The lateral distortion rate and the longitudinal distortion rate of at least part of the angles are calculated, and the preset mapping relationship is constructed according to the lateral distortion rate and the longitudinal distortion rate of at least part of the angles.

3. The method according to claim 1 or 2, characterized in that The matching of the lateral distortion rate and the longitudinal distortion rate of the measurement angle from a preset mapping relationship includes: Determining whether the measured angle is an integer angle; If the measured angle is an integer angle, a target angle that is the same as the measured angle is matched from the preset mapping relationship, and the lateral distortion rate and longitudinal distortion rate of the target angle are used as the lateral distortion rate and longitudinal distortion rate of the measured angle.

4. The method according to claim 3, characterized in that After determining whether the measured angle is the integer angle, the method further includes: If the measured angle is not the integer angle, performing a rounding operation on the measured angle to obtain a rounded angle; A target angle that is the same as the rounded angle is matched from the preset mapping relationship, and the lateral distortion rate and the longitudinal distortion rate of the target angle are used as the lateral distortion rate and the longitudinal distortion rate of the measured angle.

5. The method according to claim 1, characterized in that The determining a corrected abscissa and a corrected ordinate according to the abscissa, the ordinate, the lateral distortion rate, and the longitudinal distortion rate of the measured angle comprises: Obtaining the corrected abscissa according to the abscissa and the lateral distortion rate of the measured angle; The corrected vertical coordinate is obtained according to the vertical coordinate and the longitudinal distortion rate of the measured angle.

6. The method according to claim 5, characterized in that The obtaining the corrected abscissa according to the abscissa and the lateral distortion rate of the measured angle includes: Calculating a first difference between a first preset threshold and the lateral distortion rate of the measurement angle; The corrected horizontal coordinate is obtained according to the product of the horizontal coordinate and the first difference.

7. The method according to claim 5, characterized in that The obtaining of the corrected ordinate according to the ordinate and the longitudinal distortion rate of the measured angle includes: calculating a second difference between a second preset threshold and the longitudinal distortion rate of the measured angle; The corrected ordinate is obtained according to the product of the ordinate and the second difference.

8. The method according to claim 1, characterized in that The performing safety protection according to the corrected horizontal coordinate and the corrected vertical coordinate includes: determining a first region to which the corrected horizontal coordinate belongs and a second region to which the corrected vertical coordinate belongs; A target protection action is determined according to the first area and the second area, and security protection is performed according to the target protection action.

9. The method according to claim 8, characterized in that The determining a target protection action according to the first area and the second area, and performing security protection according to the target protection action, includes: Determining whether the first area is a preset dangerous area, or whether the second area is the preset dangerous area; If the first area is the preset dangerous area, or the second area is the preset dangerous area, immediately stop the operation of the equipment in the area to be detected; otherwise, determine whether the first area is the preset early warning area, or whether the second area is the preset early warning area; If the first area is the preset warning area, or the second area is the preset warning area, warning reminder information is generated, and a warning reminder is performed according to the warning reminder information.

10. The method according to claim 9, characterized in that After determining whether the first area is the preset warning area, or whether the second area is the preset warning area, the method further includes: If the first area is not the preset warning area, and the second area is not the preset warning area, the equipment in the area to be detected is maintained in operation.

11. The method according to claim 9, characterized in that Before determining whether the first area is a preset dangerous area, or whether the second area is the preset dangerous area, the method further includes: Obtaining the operating radius of the automation equipment in the area to be detected; The preset danger zone and the preset warning zone are determined according to the operating radius of the automation equipment.

12. The method according to claim 1, characterized in that The area to be detected includes a shielded area and a detectable area, wherein the moving object belongs to the detectable area.

13. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the security protection method according to any one of claims 1 to 12 when executing the computer program.

14. A non-volatile computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the security protection method according to any one of claims 1 to 12.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the security protection method according to any one of claims 1 to 12 are implemented.

Images