A method for measuring the swing angle of a crane load under complex background

Abstract

The invention discloses a method for measuring the swing angle of a crane load under a complex background. The method comprises the following steps: photographing a crane load and a pendulum rope in motion, establishing a mean background model under different illumination conditions; establishing an initial background model sample set, combining a foreground target extraction algorithm based on a visual background, and training to obtain a final background model sample set; extracting a foreground target; scanning pixels corresponding to the pendulum rope to obtain upper and lower endpoints of the pendulum rope, calculating the load swing angle according to the coordinates of the endpoints, extracting the pendulum rope from the complex background by using an improved foreground target extraction algorithm based on a visual background, capturing the pendulum rope in motion, and calculating the swing angle according to the endpoint coordinates of the pendulum rope pixels, thereby accelerating the algorithm running speed, ensuring the accuracy and real-time performance of the measurement method, improving the independence of the crane system by the visual measurement method, and extracting a complete foreground target for a complex background by the improved foreground target extraction algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of swing angle measurement, and in particular to a method for measuring the swing angle of a crane load under a complex background. Background Art

[0002] In recent years, bridge cranes have been widely used in construction sites, docks, industrial plants and other transportation hubs due to their fast speed and high efficiency in transporting goods. However, during transportation, due to the heavy weight of the goods and the fast speed of movement, swinging is likely to occur. If the load swing is too large, accidents can easily occur. Therefore, crane anti-sway control has become a research hotspot in the current control field.

[0003] In most of the existing crane anti-sway control systems, the mechanical devices used to measure the load swing angle are installed inside the system, such as encoders, sensors, inclinometers, etc. These measuring devices are complex to install and have high costs, which bring certain interference to the control system and affect the control efficiency. In addition, the mechanical devices have a short life and low reliability. They will become loose and aged during long-term operation, which will affect the accuracy of the measurement results. Therefore, image processing technology is introduced into the swing angle measurement, so that the measuring device is independent of the system, the system structure is simplified, and the measurement accuracy is improved. However, the traditional machine vision method for measuring the swing angle is to install the camera directly on the trolley, shoot the cycloid graph of the plane, perform threshold segmentation on the image to extract the cycloid and calculate the swing angle. The vibration of the trolley can easily cause the camera to move, causing errors. If the camera is installed outside the trolley, the two fixed light-emitting points at the top and bottom of the pendulum rope are tracked by color matching, and the swing angle is obtained by combining the pixel coordinates of the two points. Although this measurement method is fast, it requires the experimental platform to be placed in a completely dark environment, otherwise the light-emitting point cannot be accurately tracked. Using the fing edge.vi is used to find the cycloid, and caliper.vi is used to directly measure its angle with the vertical plane. The algorithm is fast and easy to implement, but other linear objects cannot interfere in the collected images. The load is identified by background modeling and the load swing angle is calculated using the Cartesian moment. The load is difficult to fully identify, and the swing angle calculation principle is complicated, which can easily cause large errors. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention proposes a method for measuring the load swing angle of a crane in a complex background.

[0006] Therefore, the technical problems solved by the present invention are: the interference brought to the control system by the mechanical device installed inside the system for measuring the load swing angle of the crane, and the traditional visual measurement method cannot be applied to the crane operating environment with a relatively complex background. Although the foreground target extraction algorithm based on visual background can effectively extract the foreground, the model training time is increased due to the presence of foreground target pixels in the initial background model sample set.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for measuring the load swing angle of a crane under a complex background, comprising:

[0008] The camera is used to capture the crane load and the swinging rope in real time, and the mean background model under different lighting conditions is established based on the video sequence;

[0009] An initial background model sample set is established based on the mean background model under different lighting conditions, and the final background model sample set is obtained through training by combining the foreground target extraction algorithm based on visual background.

[0010] The foreground target is extracted by using the final background model sample set and combining it with the foreground target extraction algorithm based on the visual background;

[0011] According to the foreground target, the corresponding pixels of the pendulum are scanned to obtain the upper and lower endpoints of the pendulum, and the load swing angle is calculated based on the coordinates of the endpoints.

[0012] As a preferred solution of the crane load swing angle measurement method under complex background described in the present invention, wherein: the mean background model under different lighting conditions is established according to the video sequence, including:

[0013] The real-time captured image is converted into a double-precision type, and the pixel mean of each frame of the video sequence is calculated to obtain a mean background model. The expression of the mean background model is:

[0014]

[0015] Among them, A n is the mean background model obtained when the nth frame is captured, N is the total number of frames, and f n ,f n-1 ,…,f n-N+1 It is a continuous N-frame image including the current frame.

[0016] As a preferred solution of the crane load swing angle measurement method under complex background described in the present invention, wherein: the mean background model under different lighting conditions is established according to the video sequence further includes:

[0017] During the modeling process, the mean background model is updated at regular intervals. The expression of the updated mean background model is:

[0018]

[0019] Among them, A n-1 is the mean background model obtained when the n-1th frame image is captured.

[0020] As a preferred solution of the crane load swing angle measurement method under complex background described in the present invention, wherein: the initial background model sample set is established according to the mean background model under different lighting conditions, including:

[0021] The mean background model under different lighting conditions is used as the initial background sample set, which is defined as:

[0022] C(m)={e1,e2,e3,…,e m-1 ,e m}

[0023] Among them, C(m) is the initial background sample set, e1,e2,e3,…,e m-1 ,e m are m initial background models.

[0024] As a preferred solution of the crane load swing angle measurement method under complex background described in the present invention, wherein: the foreground target extraction algorithm based on visual background is combined to train the final background model sample set including:

[0025] Compare the pixel points of the current frame with the pixel points in the initial background model sample set one by one, calculate the Euclidean distance between the two, and count the number of Euclidean distances less than the set segmentation threshold. When the statistical number is greater than the fixed base number, the pixel can be determined as a background point, otherwise, the pixel is a foreground point, which is expressed as:

[0026]

[0027] Among them, g(i,j) is the label of the pixel (i,j) being judged as foreground or background, R g , S g are background pixels and foreground pixels respectively, x is the number of pixels in the current frame whose Euclidean distance with the background model pixels in the initial background model sample set is less than the set segmentation threshold, and k is a fixed cardinality.

[0028] As a preferred solution of the crane load swing angle measurement method under complex background described in the present invention, wherein: the foreground target extraction method based on visual background is combined to train the final background model sample set, and further comprises:

[0029] At the same time, the sample models in the initial background model sample set are updated. If a pixel point in the current frame is judged to be a background pixel point, any sample model in the initial background model sample set is randomly selected, and the grayscale value of the corresponding pixel point is changed to the grayscale value of the background point, and any adjacent pixel point is randomly changed again, and its grayscale value is also changed to the grayscale value of the background point. During the updating process, if the complete detection target can be extracted, the updating is stopped and the final background model is obtained.

[0030] As a preferred solution of the crane load swing angle measurement method under complex background described in the present invention, wherein: the extraction of foreground targets by using the final background model sample set and combining the foreground target extraction algorithm based on visual background includes:

[0031] Compare the current frame pixel with the pixel in the final background model sample set one by one, calculate the Euclidean distance between the two, and count the number of Euclidean distances less than the set segmentation threshold. When the number is less than the fixed base number, the pixel is judged as a foreground point, otherwise, the pixel is a background point, which is expressed as:

[0032]

[0033] Among them, g(i,j) is the label of the pixel (i,j) being judged as foreground or background, R g , S g are background pixels and foreground pixels respectively, x is the number of pixels in the current frame whose Euclidean distance with the background model pixels in the initial background model sample set is less than the set segmentation threshold, and k is a fixed cardinality.

[0034] As a preferred solution of the crane load swing angle measurement method under complex background described in the present invention, wherein: scanning the corresponding pixels of the swing rope according to the foreground target, obtaining the upper and lower endpoints of the swing rope, and calculating the load swing angle according to the coordinates of the endpoints includes:

[0035] In the process of swing angle calculation, the region of interest of the original image is intercepted, and the intercepted image is subjected to median filtering to eliminate noise interference in the image. The image is binarized and the bwareaopen algorithm is used to delete the region in the image until the white area in the image is completely swing rope.

[0036] As a preferred solution of the crane load swing angle measurement method under complex background described in the present invention, wherein: scanning the corresponding pixels of the swing rope according to the foreground target, obtaining the upper and lower endpoints of the swing rope, and calculating the load swing angle according to the coordinates of the endpoints also includes:

[0037] Scan the top and bottom rows of the image, determine the image blocks with certain pixel grayscale values, and use the centers of the two areas as the upper and lower endpoints of the unscanned cycloid image area, thereby obtaining two pixel coordinates (u1, v1) and (u2, v2), and then convert the pixel coordinates into actual coordinates:

[0038] U P =(u P -u0) / d u

[0039] V P =(v P -v0) / d v

[0040] P=1,2

[0041] Among them, (U P ,V P ) is the actual coordinate after transformation, (u0,v0) is the origin of image coordinate, d u ,d v They are the number of pixels per row and column per unit length of the camera respectively.

[0042] As a preferred solution of the crane load swing angle measurement method under complex background described in the present invention, wherein: scanning the corresponding pixels of the swing rope according to the foreground target, obtaining the upper and lower endpoints of the swing rope, and calculating the load swing angle according to the coordinates of the endpoints also includes:

[0043] Perform distortion compensation on the actual coordinates:

[0044]

[0045] Among them, (U 1P ,V 1P ) is the actual coordinate after compensation, α is the fixed parameter of the camera, and finally, the swing angle is calculated according to the actual coordinate after compensation:

[0046]

[0047]

[0048] Among them, (U 11 ,V 11 ) is the actual coordinate of the upper end point of the pendulum, (U 12 ,V 12 ) is the actual coordinate of the lower end point of the pendulum.

[0049] The beneficial effects of the present invention are as follows: the improved foreground target extraction algorithm based on visual background is used to extract the pendulum from the complex background, thereby capturing the pendulum in motion, and the swing angle is calculated by the endpoint coordinates of the pendulum pixels, thereby accelerating the algorithm operation speed and ensuring the accuracy and real-time performance of the measurement method. The visual measurement method improves the independence of the crane system, and the improved foreground target extraction algorithm based on visual background can also extract complete foreground targets for complex backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0051] Figure 1 is a flow chart of the method of the present invention;

[0052] Figure 2 A visual measurement system of the present invention;

[0053] Figure 3 This is a comparison chart of the swing angle finally calculated and the swing angle measured by the encoder;

[0054] Figure 4 This is a diagram to verify the real-time performance of the visual measurement system. DETAILED DESCRIPTION

[0055] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0056] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0057] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0058] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0059] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0060] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0061] Example 1

[0062] Reference Figures 1-2 This embodiment provides a method for measuring a crane load swing angle in a complex background, including:

[0063] S1: Use the camera to shoot the crane load and the swing rope in real time, and build the mean background model under different lighting conditions based on the video sequence.

[0064] The real-time captured image is converted into a double-precision type, and the pixel mean of each frame of the video sequence is calculated to obtain a mean background model. The expression of the mean background model is:

[0065]

[0066] Among them, A n is the mean background model obtained when the nth frame is captured, N is the total number of frames, and f n ,f n-1 ,…,f n-N+1 It is a continuous N-frame image including the current frame.

[0067] During the modeling process, since the external environment such as light can change at any time, the mean background model is updated at regular intervals. The expression of the updated mean background model is:

[0068]

[0069] Among them, A n-1 is the mean background model obtained when the n-1th frame image is captured.

[0070] S2: Establish an initial background model sample set based on the mean background model under different lighting conditions, and combine it with the foreground target extraction algorithm based on visual background to train the final background model sample set.

[0071] The mean background model under different lighting conditions is used as the initial background sample set, which is defined as:

[0072] C(m)={e1,e2,e3,…,e m-1 ,e m}

[0073] Among them, C(m) is the initial background sample set, e1,e2,e3,…,e m-1 ,e m are m initial background models.

[0074] Compare the pixel points of the current frame with the pixel points in the initial background model sample set one by one, calculate the Euclidean distance between the two, and count the number of Euclidean distances less than the set segmentation threshold. When the statistical number is greater than the fixed base number, the pixel can be determined as a background point, otherwise, the pixel is a foreground point, which is expressed as:

[0075]

[0076] Among them, g(i,j) is the label of the pixel (i,j) being judged as foreground or background, R g , S g are background pixels and foreground pixels respectively, x is the number of pixels in the current frame whose Euclidean distance with the background model pixels in the initial background model sample set is less than the set segmentation threshold, and k is a fixed cardinality.

[0077] At the same time, the sample models in the initial background model sample set are updated. If a pixel point in the current frame is judged to be a background pixel point, any sample model in the initial background model sample set is randomly selected, and the grayscale value of the corresponding pixel point is changed to the grayscale value of the background point. Based on the characteristic that adjacent pixels of pixels in the image have similar distribution, any adjacent pixel point is randomly changed again, and its grayscale value is also changed to the grayscale value of the background point. During the updating process, if the complete detection target can be extracted, that is, the foreground point is relatively complete, the updating is stopped and the final background model is obtained. In the process of training the final background model, the specific total number of modeling frames and the segmentation threshold can be determined by the modeling effect. The number of pixels per unit length of each row and column of the camera and its fixed parameters will not affect the final calculated swing angle value.

[0078] S3: Extract foreground targets using the final background model sample set and the foreground target extraction algorithm based on visual background.

[0079] Compare the current frame pixel with the pixel in the final background model sample set one by one, calculate the Euclidean distance between the two, and count the number of Euclidean distances less than the set segmentation threshold. When the number is less than the fixed base number, the pixel is judged as a foreground point, otherwise, the pixel is a background point, which is expressed as:

[0080]

[0081] Among them, g(i,j) is the label of the pixel (i,j) being judged as foreground or background, R g , S g are background pixels and foreground pixels respectively, x is the number of pixels in the current frame whose Euclidean distance with the background model pixels in the initial background model sample set is less than the set segmentation threshold, and k is a fixed cardinality.

[0082] S4: Scan the corresponding pixels of the pendulum according to the foreground target, obtain the upper and lower endpoints of the pendulum, and calculate the load swing angle according to the coordinates of the endpoints.

[0083] In the process of swing angle calculation, since the movement range of the pendulum only occupies one-third of the entire image, the area of ​​interest of the original image is intercepted. After interception, the noise interference in the non-interested area can be avoided, the processing speed of the algorithm can be improved, and the real-time requirements of the measurement system can be met. The intercepted image is median filtered to eliminate the noise interference in the image and binarize the image. In order to prevent the existence of a small number of non-swing rope areas in the image, the bwareaopen algorithm is used to delete the area in the image and delete the small connected areas in the image until the white area in the image is completely the swing rope.

[0084] In order to prevent calculation errors, scan the top and bottom rows of the image, determine the image blocks with pixel grayscale values ​​of 255, and use the centers of the two areas as the upper and lower endpoints of the unscanned cycloid image area, thereby obtaining two pixel coordinates (u1, v1) and (u2, v2), and then convert the pixel coordinates into actual coordinates:

[0085] U P =(u P -u0) / d u

[0086] V P =(v P -v0) / d v

[0087] P=1,2

[0088] Among them, (U P ,V P ) is the actual coordinate after transformation, (u0,v0) is the origin of image coordinate, d u ,d v They are the number of pixels per row and column per unit length of the camera respectively.

[0089] Since the captured image of the rope swing is easily affected by lens distortion, the actual coordinates are compensated for distortion:

[0090]

[0091] Among them, (U 1P ,V 1P ) is the actual coordinate after compensation, α is the fixed parameter of the camera, and finally, the swing angle is calculated according to the actual coordinate after compensation:

[0092]

[0093]

[0094] Among them, (U 11 ,V 11 ) is the actual coordinate of the upper end point of the pendulum, (U 12 ,V 12 ) is the actual coordinate of the lower end point of the pendulum.

[0095] Example 2

[0096] Reference Figures 1 to 4 This embodiment provides a method for measuring the swing angle of a crane load in a complex background. In order to verify and illustrate the technical effect adopted in this method, the swing angle value measured by the encoder of the physical device is compared with the swing angle value calculated by the measurement method of the present invention to verify the real effect of the present invention.

[0097] like Figure 2 As shown, in order to verify the accuracy and real-time performance of the measurement results of the present invention, a high-definition CMOS camera is used, which can collect 30 frames of images per second. It is fixed in front of the crane system, and the camera position is adjusted so that it can capture the complete swing rope and obtain image frames in real time. The load is allowed to swing back and forth in the vertical plane. Figure 1 The crane load swing angle measurement method shown measures the swing angle in real time and plots the measurement results into a curve.

[0098] The improved foreground target extraction algorithm based on visual background can train an accurate background model and completely extract the foreground target, such as Figure 3 As shown in FIG. 1 , the black dotted line is the swing angle value calculated by the algorithm of the present invention, and the black solid line is the encoder measurement value. It can be seen that the visual measurement result is almost the same as the encoder measurement result, which verifies the accuracy of the measurement method of the present invention. Figure 4 As shown, the solid points represent the measurement sampling points. 240 swing angle values ​​can be calculated within 10 seconds, the frequency reaches 24 Hz, and the change of data size over time can show the change of angle when the load swings, which verifies the real-time performance of the measurement method of the present invention.

[0099] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in an assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed ASIC for this purpose.

[0100] Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that may be executed by one or more processors.

[0101] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or part thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques described in the present invention, the present invention also includes the computer itself. The computer program can be applied to input data to perform the functions described herein, thereby converting the input data to generate output data stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on a display.

[0102] As used in this application, the terms "component", "module", "system", etc. are intended to refer to a computer-related entity, which can be hardware, firmware, a combination of hardware and software, software, or software in operation. For example, a component can be, but is not limited to: a process running on a processor, a processor, an object, an executable file, a thread in execution, a program, and / or a computer. As an example, an application running on a computing device and the computing device can both be components. One or more components can exist in a process and / or thread in execution, and a component can be located in a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures thereon. These components can communicate in a local and / or remote process manner, such as according to a signal having one or more data packets (e.g., data from a component that interacts with another component in a local system, a distributed system, and / or interacts with other systems in a signal manner through a network such as the Internet).

[0103] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for measuring the load swing angle of a crane under complex background, characterized in that: include, The camera is used to capture the crane load and the swinging rope in real time, and the mean background model under different lighting conditions is established based on the video sequence; An initial background model sample set is established based on the mean background model under different lighting conditions, and the final background model sample set is obtained through training by combining the foreground target extraction algorithm based on visual background. The foreground target is extracted by using the final background model sample set and combining it with the foreground target extraction algorithm based on the visual background; According to the foreground target, scan the corresponding pixels of the pendulum rope to obtain the upper and lower endpoints of the pendulum rope, and calculate the load swing angle according to the coordinates of the endpoints; According to the foreground target, the corresponding pixels of the pendulum rope are scanned to obtain the upper and lower endpoints of the pendulum rope. The load swing angle is calculated according to the coordinates of the endpoints, which also includes: Scan the top and bottom rows of the image, determine the image blocks with certain pixel grayscale values, and use the centers of the two areas as the upper and lower endpoints of the unscanned cycloid image area, thereby obtaining two pixel coordinates (u1, v1) and (u2, v2), and then convert the pixel coordinates into actual coordinates: U P =(u P -u0) / d u V P =(v P -v0) / d v P＝1,2 Among them, (U P ,V P ) is the actual coordinate after transformation, (u0,v0) is the origin of image coordinate, d u d v are the number of pixels per row and column per unit length of the camera respectively; According to the foreground target, the corresponding pixels of the pendulum rope are scanned to obtain the upper and lower endpoints of the pendulum rope. The load swing angle is calculated according to the coordinates of the endpoints, which also includes: Perform distortion compensation on the actual coordinates: Among them, (U 1P ,V 1P ) is the actual coordinate after compensation, α is the fixed parameter of the camera, and finally, the swing angle is calculated according to the actual coordinate after compensation: Among them, (U 11 ,V 11 ) is the actual coordinate of the upper end point of the pendulum, (U 12 ,V 12 ) is the actual coordinate of the lower end point of the pendulum.

2. The method for measuring the crane load swing angle under complex background according to claim 1, characterized in that: The method of establishing a mean background model under different illumination conditions according to the video sequence comprises: The real-time captured image is converted into a double-precision type, and the pixel mean of each frame of the video sequence is calculated to obtain a mean background model. The expression of the mean background model is: Among them, A n is the mean background model obtained when the nth frame is captured, N is the total number of frames, and f n ,f n-1 ,…,f n-N+1 It is a continuous N-frame image including the current frame.

3. The method for measuring the crane load swing angle under complex background according to claim 2, characterized in that: The method of establishing a mean background model under different illumination conditions according to the video sequence further includes: During the modeling process, the mean background model is updated at regular intervals. The expression of the updated mean background model is: Among them, A n-1 is the mean background model obtained when the n-1th frame image is captured.

4. The method for measuring the crane load swing angle under complex background according to claim 3, characterized in that: The method of establishing an initial background model sample set according to the mean background model under different illumination conditions comprises: The mean background model under different lighting conditions is used as the initial background sample set, which is defined as: C(m)={e1,e2,e3,…,e m-1 ,e m } Among them, C(m) is the initial background sample set, e1,e2,e3,…,e m-1 ,e m are m initial background models.

5. The method for measuring the crane load swing angle under complex background according to claim 4, characterized in that: The final background model sample set obtained by training the foreground target extraction algorithm based on visual background includes: Compare the pixel points of the current frame with the pixel points in the initial background model sample set one by one, calculate the Euclidean distance between the two, and count the number of Euclidean distances less than the set segmentation threshold. When the statistical number is greater than the fixed base number, the pixel can be determined as a background point, otherwise, the pixel is a foreground point, which is expressed as: Among them, g(i,j) is the label of the pixel (i,j) being judged as foreground or background, R g , S g are background pixels and foreground pixels respectively, x is the number of pixels in the current frame whose Euclidean distance with the background model pixels in the initial background model sample set is less than the set segmentation threshold, and k is a fixed cardinality.

6. The method for measuring the crane load swing angle under complex background according to claim 5, characterized in that: The method for extracting foreground objects based on visual background is combined with the training to obtain the final background model sample set, which further includes: At the same time, the sample models in the initial background model sample set are updated. If a pixel point in the current frame is judged to be a background pixel point, any sample model in the initial background model sample set is randomly selected, and the grayscale value of the corresponding pixel point is changed to the grayscale value of the background point, and any adjacent pixel point is randomly changed again, and its grayscale value is also changed to the grayscale value of the background point. During the updating process, if the complete detection target can be extracted, the updating is stopped and the final background model is obtained.

7. The method for measuring the crane load swing angle under complex background according to claim 6, characterized in that: The method of extracting the foreground target by using the final background model sample set and combining the foreground target extraction algorithm based on the visual background includes: Compare the current frame pixel with the pixel in the final background model sample set one by one, calculate the Euclidean distance between the two, and count the number of Euclidean distances less than the set segmentation threshold. When the number is less than the fixed base number, the pixel is judged as a foreground point, otherwise, the pixel is a background point, which is expressed as: Among them, g(i,j) is the label of the pixel (i,j) being judged as foreground or background, R g , S g are background pixels and foreground pixels respectively, x is the number of pixels in the current frame whose Euclidean distance with the background model pixels in the initial background model sample set is less than the set segmentation threshold, and k is a fixed cardinality.

8. The method for measuring the crane load swing angle under complex background according to claim 7, characterized in that: The method of scanning the pixels corresponding to the pendulum rope according to the foreground target, obtaining the upper and lower endpoints of the pendulum rope, and calculating the load swing angle according to the coordinates of the endpoints includes: In the process of swing angle calculation, the region of interest of the original image is intercepted, and the intercepted image is subjected to median filtering to eliminate noise interference in the image. The image is binarized and the bwareaopen algorithm is used to delete the region in the image until the white area in the image is completely swing rope.

Images