Methods, apparatus, processors, and engineering equipment for identifying outrigger support status.

Abstract

This invention provides a method, apparatus, processor, and engineering equipment for identifying the support state of outriggers, belonging to the field of engineering machinery. The method for identifying the support state of outriggers is applied to engineering equipment including outriggers. An image acquisition device is installed on the outrigger to acquire image sequences. The method includes: acquiring the position information of target feature points in the image sequence; determining a valid point set based on the position information of the target feature points, wherein the valid point set is a collection of valid target feature points, and the displacement change along the height direction between adjacent valid target feature points reaches a preset displacement change amount; determining the cumulative displacement change of the valid target feature points along the height direction based on the position information of the valid target feature points; and determining the support state of the outrigger based on the cumulative displacement change amount. This invention can reduce reliance on operators and improve equipment operation safety.

Description

Technical Field

[0001] This invention relates to the field of engineering machinery technology, and more specifically to a method, apparatus, processor, and engineering equipment for identifying the support status of outriggers. Background Technology

[0002] In the field of construction machinery, outriggers are typically deployed before construction operations to ensure safety. If the ground bearing capacity of the outriggers is insufficient, they may collapse, potentially causing the equipment to tip over. Currently, the outrigger support status is usually determined visually by the operator, leading to an over-reliance on human intervention. Summary of the Invention

[0003] The purpose of this invention is to provide a method, apparatus, processor, engineering equipment, and storage medium for identifying the support status of outriggers, in order to solve the problem of excessive reliance on operators in the prior art.

[0004] To achieve the above objectives, a first aspect of the present invention provides a method for identifying the support state of a leg, applied to engineering equipment including a leg, wherein an image acquisition device is provided on the leg, the image acquisition device being used to acquire image sequences, and the method comprising:

[0005] Obtain the location information of target feature points in an image sequence;

[0006] The effective point set is determined based on the location information of the target feature points. The effective point set is the set of effective target feature points among the target feature points. The displacement change along the height direction between adjacent effective target feature points reaches the preset displacement change amount.

[0007] The cumulative displacement change of the effective target feature points along the height direction is determined based on the position information of the effective target feature points;

[0008] The support status of the outriggers is determined based on the cumulative displacement change.

[0009] In this embodiment of the invention, determining the support state of the outrigger based on the cumulative displacement change includes: comparing the cumulative displacement change with a preset cumulative displacement change; if the cumulative displacement change is greater than the preset cumulative displacement change, determining the support state of the outrigger as outrigger collapse or outrigger elevation; if the cumulative displacement change is not greater than the preset cumulative displacement change, determining the support state of the outrigger as normal outrigger support.

[0010] In this embodiment of the invention, the number of valid target feature points in the valid point set is not greater than a preset upper limit threshold.

[0011] In this embodiment of the invention, obtaining the location information of target feature points in an image sequence includes: obtaining the location information of target feature points in an image sequence within a preset time period.

[0012] In this embodiment of the invention, the number of target feature points is m, where m is greater than 1. Determining the effective point set based on the position information of the target feature points includes: determining the displacement change of the m target feature points in the current frame image relative to the corresponding m target feature points in the previous effective frame image along the height direction, to obtain m displacement changes, wherein the effective image is an image in the image sequence where all m target feature points are located in the effective point set; determining the number h of effective displacement changes that reach a preset displacement change among the m displacement changes; determining the ratio of the number h of effective displacement changes to the number m of displacement changes, to obtain a first ratio; if the first ratio is greater than the first preset ratio, adding the m target feature points in the current frame image to the effective point set; if the first ratio is less than or equal to the first preset ratio, not adding the m target feature points in the current frame image to the effective point set.

[0013] In this embodiment of the invention, the number of target feature points is single; determining the cumulative displacement change of the effective target feature points along the height direction based on the position information of the effective target feature points includes: determining the displacement change of the last effective target feature point in the effective point set and the first effective target feature point in the effective point set along the height direction to obtain the cumulative displacement change.

[0014] In this embodiment of the invention, the number of target feature points is m, where m is greater than 1; determining the cumulative displacement change of the effective target feature points along the height direction based on the position information of the effective target feature points includes: determining the displacement change of the m effective target feature points in the last group of the effective point set and the m effective target feature points in the first group of the effective point set along the height direction, respectively, to obtain m cumulative displacement changes.

[0015] In this embodiment of the invention, determining the support state of the outrigger based on the cumulative displacement change includes: determining the number k of effective cumulative displacement changes greater than a preset cumulative displacement change among m cumulative displacement changes; determining the ratio of the number k of effective cumulative displacement changes to the number m of cumulative displacement changes to obtain a second ratio; if the second ratio is greater than a second preset ratio, determining the support state of the outrigger as outrigger collapse or outrigger lifting; if the second ratio is less than or equal to the second preset ratio, determining the support state of the outrigger as normal outrigger support.

[0016] A second aspect of the present invention provides a processor configured to execute the method described above for identifying the support state of a leg.

[0017] A third aspect of this invention provides a device for identifying the support state of a leg, applied to engineering equipment including a leg. The leg is equipped with an image acquisition device for acquiring image sequences. The device includes:

[0018] The location acquisition module is used to acquire the location information of target feature points in the image sequence;

[0019] The effective point set determination module is used to determine the effective point set based on the position information of the target feature points. The effective point set is the set of effective target feature points among the target feature points, and the displacement change along the height direction between adjacent effective target feature points reaches a preset displacement change amount.

[0020] The cumulative displacement determination module is used to determine the cumulative displacement change of the effective target feature points along the height direction based on the position information of the effective target feature points;

[0021] The support status determination module is used to determine the support status of the outriggers based on the cumulative displacement change.

[0022] A fourth aspect of the present invention provides an engineering device, comprising: a support leg; an image acquisition device disposed on the support leg for acquiring image sequences; and a processor according to the above or a device according to the above for identifying the support state of the support leg.

[0023] A fifth aspect of the present invention provides a machine-readable storage medium on which a program or instruction is stored, which, when executed by a processor, implements the method described above for identifying the support state of a leg.

[0024] The above technical solution involves setting up an image acquisition device on the outrigger and acquiring the position information of target feature points in the image sequence. Based on the position information of the target feature points, a set of effective points is determined, where the set of effective target feature points is the collection of effective target feature points. The displacement change along the height direction between adjacent effective target feature points reaches a preset displacement change amount. Then, based on the position information of the effective target feature points, the cumulative displacement change of the effective target feature points along the height direction is determined, thereby determining the support state of the outrigger based on the cumulative displacement change amount. The above technical solution eliminates the need for operators to visually identify the support status of the outriggers, reducing reliance on operators and improving equipment safety. Based on the positional information of target feature points, effective target feature points that have undergone effective movement in the height direction are selected. The presence of group movement among these effective feature points can be determined by analyzing their cumulative displacement along the height direction. Since the image acquisition device is mounted on the outriggers, the support status of the outriggers can be determined based on the overall movement direction of the effective target feature points in the height direction within the image sequence. This achieves accurate identification of the outrigger support status and improves the accuracy of outrigger support status identification.

[0025] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0026] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0027] Figure 1 The schematic diagram illustrates a flowchart of a method for identifying the support state of a support leg according to an embodiment of the present invention;

[0028] Figure 2 This schematic diagram illustrates the framework of a system for identifying the support state of outriggers according to an embodiment of the present invention.

[0029] Figure 3 A schematic diagram of an engineering device according to an embodiment of the present invention is shown;

[0030] Figure 4 This schematic diagram illustrates the image coordinate system and the positional changes of target feature points in one embodiment of the present invention.

[0031] Figure 5 The diagram illustrates a structural block diagram of a device for identifying the support state of a leg, according to an embodiment of the present invention. Detailed Implementation

[0032] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0033] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0034] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0035] Figure 1 The illustration schematically shows a flowchart of a method for identifying the support state of an outrigger according to an embodiment of the present invention. Figure 1 As shown, in this embodiment of the invention, a method for identifying the support state of a leg is provided, applied to engineering equipment including legs. An image acquisition device is mounted on the leg to acquire image sequences. Taking the application of this method to a processor as an example, the method may include the following steps:

[0036] Step S102: Obtain the location information of the target feature points in the image sequence.

[0037] Step S104: Determine the effective point set based on the position information of the target feature points. The effective point set is the set of effective target feature points among the target feature points, and the displacement change along the height direction between adjacent effective target feature points reaches the preset displacement change amount.

[0038] Step S106: Determine the cumulative displacement change of the effective target feature points along the height direction based on the position information of the effective target feature points.

[0039] Step S108: Determine the support status of the outriggers based on the cumulative displacement change.

[0040] Image sequences, or video sequences, are essentially consecutive image frames. Target feature points are pre-defined feature points in the image, such as corner points. Corners are locations with large grayscale gradients in the image. Positional information can include pixel coordinates. Valid target feature points are those that have undergone effective movement or displacement along the height direction; these are the feature points that can be retained for later use in determining the support state of the outrigger. The valid point set is a collection of valid target feature points. For each valid target feature point in the valid point set, the displacement change along the height direction between adjacent valid target feature points reaches a preset displacement change amount. For example, the displacement change along the height direction between the second and first valid target feature points in the valid point set reaches the preset displacement change amount, the displacement change along the height direction between the third and second valid target feature points reaches the preset displacement change amount, and so on. The preset displacement change amount is a pre-set threshold for displacement change, and its specific value can be represented by, for example, the distance between a preset number of pixels. The cumulative displacement change refers to the cumulative displacement change between valid target feature points in the effective point set, specifically the displacement change between the last and first valid target feature points in the effective point set. The cumulative displacement change along the height direction is the same as the displacement change along the height direction between the last and first valid target feature points in the effective point set. The support state of the outrigger can include outrigger collapse, outrigger elevation, or normal outrigger support. Understandably, the displacement change in this embodiment is a non-negative number, i.e., the absolute value. Furthermore, the image acquisition device is mounted on the outrigger. In other words, the image acquisition device and the outrigger remain relatively stationary. When the outrigger is normally supported, the video image captured by the image acquisition device (such as a camera) is static. When the outrigger collapses or rises, the video image captured by the image acquisition device (such as a camera) also moves synchronously. For example, assuming the outrigger is slowly collapsing downwards, the camera mounted on the outrigger will also move downwards synchronously. The fixed target P (i.e., the target feature point, such as a fixed point on the wall) in the video sequence captured by the camera will move upwards from position (x1, y1) to position (x2, y2) in the image.

[0041] Specifically, the processor can acquire image sequences captured by an image acquisition device (e.g., a camera), thereby extracting feature points from each frame of the image sequence and matching feature points in the image sequence, and obtaining the position information of target feature points in the image sequence. Then, it can determine a set of valid points based on the position information of the target feature points. For example, the processor can determine the displacement change of the current target feature point and the previous valid target feature point in the set of valid points along the height direction based on the position information of the target feature points, and compare the displacement change with a preset displacement change. If the displacement change is greater than or equal to the preset displacement change, the current target feature point can be determined as a valid target feature point and added to the set of valid points. Otherwise, it is not added to the set of valid points, for example, the current target feature point can be discarded. Finally, the displacement change between adjacent valid target feature points in the set of valid points along the height direction reaches the preset displacement change. After determining the effective point set, the processor can determine the cumulative displacement change of the effective target feature points along the height direction based on the position information of the effective target feature points. That is, the processor can determine the displacement change between the last effective target feature point and the first effective target feature point along the height direction based on the position information of the first and last effective target feature points in the effective point set, and determine the support state of the outrigger based on the cumulative displacement change. For example, according to a pre-determined correspondence table between cumulative displacement change and support state, the processor can look up the corresponding table based on the calculated cumulative displacement change to determine the support posture of the outrigger corresponding to the cumulative displacement change. Understandably, the larger the cumulative displacement change, the greater the possibility that the support posture of the outrigger is collapse or lifting, and the smaller the cumulative displacement change, the greater the possibility that the support posture of the outrigger is normal support.

[0042] The above-described method for identifying the support state of outriggers involves setting up an image acquisition device on the outrigger and acquiring the position information of target feature points in an image sequence. Based on the position information of the target feature points, a set of valid points is determined, wherein the set of valid target feature points is the collection of valid target feature points. The displacement change along the height direction between adjacent valid target feature points reaches a preset displacement change amount. Then, based on the position information of the valid target feature points, the cumulative displacement change of the valid target feature points along the height direction is determined, thereby determining the support state of the outrigger based on the cumulative displacement change amount. The above technical solution eliminates the need for operators to visually identify the support status of the outriggers, reducing reliance on operators and improving equipment safety. Based on the positional information of target feature points, effective target feature points that have undergone effective movement in the height direction are selected. The presence of group movement among these effective feature points can be determined by analyzing their cumulative displacement along the height direction. Since the image acquisition device is mounted on the outriggers, the support status of the outriggers can be determined based on the overall movement direction of the effective target feature points in the height direction within the image sequence. This achieves accurate identification of the outrigger support status and improves the accuracy of outrigger support status identification.

[0043] In one embodiment, determining the support state of the outrigger based on the cumulative displacement change includes: comparing the cumulative displacement change with a preset cumulative displacement change; if the cumulative displacement change is greater than the preset cumulative displacement change, determining the support state of the outrigger as outrigger collapse or outrigger elevation; if the cumulative displacement change is not greater than the preset cumulative displacement change, determining the support state of the outrigger as normal outrigger support.

[0044] It is understandable that the preset cumulative displacement change is a pre-set threshold for the cumulative displacement change.

[0045] Specifically, the processor can compare the cumulative displacement change of the effective target feature point along the height direction with a preset cumulative displacement change. When it is determined that the cumulative displacement change is greater than the preset cumulative displacement change, the processor can determine that the support state of the outrigger is either outrigger collapse or outrigger lifting. When it is determined that the cumulative displacement change is not greater than the preset cumulative displacement change, the processor can determine that the support state of the outrigger is normal outrigger support.

[0046] In one embodiment, the number of valid target feature points in the valid point set is no greater than a preset upper limit threshold.

[0047] It is understandable that the preset upper limit threshold is the maximum number of valid target feature points in the pre-set valid point set, such as 8 or 10.

[0048] Understandably, if the number of valid target feature points in the valid point set exceeds a preset upper limit threshold, the processor can clear the valid point set, i.e., redetermine the valid target feature points in the valid point set. In other words, when determining the cumulative displacement change of valid target feature points along the height direction based on their position information, taking a preset upper limit threshold of 8 as an example, the processor can determine the cumulative displacement change of valid target feature points along the height direction based on the position information of valid target feature points in a valid point set containing a maximum of 8 valid target feature points.

[0049] In this embodiment of the application, the number of valid target feature points in the effective point set is limited, that is, a preset upper limit threshold is set in advance, which can speed up the recognition speed of the leg support state and further improve the recognition efficiency and recognition accuracy of the leg support state.

[0050] In one embodiment, obtaining the location information of target feature points in an image sequence includes: obtaining the location information of target feature points in an image sequence within a preset time period.

[0051] It's understandable that the preset time period is a pre-set time period, such as 5 seconds.

[0052] Understandably, the processor can obtain the location information of target feature points in an image sequence within a preset time period, and then determine the effective point set based on the location information of target feature points in the image sequence within the preset time period, thereby proceeding with subsequent steps.

[0053] In this embodiment of the application, the support state of the outrigger is identified based on the location information of the target feature points in the image sequence within a preset time period. This can shorten the identification time of the outrigger support state and further improve the identification efficiency and accuracy of the outrigger support state.

[0054] In one embodiment, the number of target feature points is m, where m is greater than 1. Determining the effective point set based on the position information of the target feature points includes: determining the displacement change along the height direction of the m target feature points in the current frame image relative to the corresponding m target feature points in the previous effective frame image, to obtain m displacement changes, wherein the effective image is an image in the image sequence where all m target feature points are located in the effective point set; determining the number h of effective displacement changes that reach a preset displacement change among the m displacement changes; determining the ratio of the number h of effective displacement changes to the number m of displacement changes, to obtain a first ratio; if the first ratio is greater than a first preset ratio, adding the m target feature points in the current frame image to the effective point set; if the first ratio is less than or equal to the first preset ratio, not adding the m target feature points in the current frame image to the effective point set.

[0055] It can be understood that the number of target feature points in each frame of an image sequence can be single or multiple (i.e., m, m>1). Taking multiple as an example, the image sequence can be divided into valid images and invalid images. A valid image is an image in which all m target feature points in the image sequence are located in the valid point set, while an invalid image is an image in which none of the m target feature points are located in the valid point set. Furthermore, valid images can be retained, and invalid images can be discarded. The effective displacement change is the displacement change greater than or equal to a preset displacement change among the m displacement changes, i.e., the displacement change that has resulted in effective displacement. The first ratio is the ratio of the number of effective displacement changes h to the total number of displacement changes m. The first preset ratio is a pre-set threshold related to the number of displacement changes, such as 70% or 75%. That is, if the ratio of the number of m target feature points in an image to the corresponding m target feature points in the previous valid image along the height direction reaches a preset displacement change greater than the first preset ratio (e.g., 70%), then all m target feature points in that image can be added to the valid point set. The current frame image is the image frame at the current moment.

[0056] Specifically, the processor can determine the displacement changes of m target feature points in the current frame image relative to the corresponding m target feature points in the previous valid frame image along the height direction based on the position information of the target feature points. This yields m displacement changes, which are then compared with preset displacement changes to determine the number h of effective displacement changes that reach the preset displacement changes. The processor can then determine the ratio of the number h of effective displacement changes to the total number m of displacement changes, obtaining a first ratio. This first ratio is compared with a first preset ratio. If the first ratio is greater than the first preset ratio, all m target feature points in the current frame image are added to the valid point set. Conversely, if the first ratio is less than or equal to the first preset ratio, the processor does not add the m target feature points in the current frame image to the valid point set.

[0057] In this embodiment of the application, when there are multiple target feature points, the first ratio of the number of effective displacement changes h to the total number of displacement changes m is compared with a first preset ratio. If the first ratio is greater than the first preset ratio, it indicates that the target feature points have a collective movement, thereby allowing the image acquisition device, such as a camera, to be effectively moved.

[0058] In one embodiment, the number of target feature points is single; determining the cumulative displacement change of the effective target feature points along the height direction based on the position information of the effective target feature points includes: determining the displacement change of the last effective target feature point in the effective point set and the first effective target feature point in the effective point set along the height direction to obtain the cumulative displacement change.

[0059] Specifically, the processor can subtract the coordinates of the last valid target feature point in the set of valid points in the height direction from the coordinates of the first valid target feature point in the set of valid points in the height direction, and take the absolute value of the subtraction result to obtain the cumulative displacement change.

[0060] In one embodiment, the number of target feature points is m, where m is greater than 1; determining the cumulative displacement change of the effective target feature points along the height direction based on the position information of the effective target feature points includes: determining the displacement change of the m effective target feature points in the last group of the effective point set and the m effective target feature points in the first group of the effective point set along the height direction, respectively, to obtain m cumulative displacement changes.

[0061] It can be understood that when the number of target feature points is multiple, i.e., m (m>1), the m effective target feature points of the first group in the effective point set are the m target feature points in the first frame image added to the effective point set, and the m effective target feature points of the last group in the effective point set are the m target feature points in the last frame image added to the effective point set. It can be understood that the m effective target feature points of the first group and the m effective target feature points of the last group are mutually matched target feature points, and the m effective target feature points of each group in the effective point set are all matched target feature points.

[0062] Specifically, the processor can determine the displacement changes along the height direction of the m effective target feature points in the last group of the effective point set and the m effective target feature points in the first group of the effective point set, respectively. That is, the processor subtracts the coordinates of the m effective target feature points in the height direction of the two groups respectively, and takes the absolute value of the subtraction result to obtain the m cumulative displacement changes.

[0063] In one embodiment, determining the support state of the outrigger based on the cumulative displacement change includes: determining the number k of effective cumulative displacement changes greater than a preset cumulative displacement change among m cumulative displacement changes; determining the ratio of the number k of effective cumulative displacement changes to the number m of cumulative displacement changes to obtain a second ratio; if the second ratio is greater than a second preset ratio, determining the support state of the outrigger as outrigger collapse or outrigger lifting; if the second ratio is less than or equal to the second preset ratio, determining the support state of the outrigger as normal outrigger support.

[0064] It can be understood that the preset cumulative displacement change is a pre-set threshold for cumulative displacement change. The effective cumulative displacement change is the cumulative displacement change that is greater than the preset cumulative displacement change. The second ratio is the ratio of the number of effective cumulative displacement changes k to the total number of cumulative displacement changes m. Understandably, m target feature points correspond to m effective target feature points, and m effective target feature points correspond to m cumulative displacement changes. The second preset ratio is a pre-set proportional threshold related to the number of cumulative displacement changes.

[0065] Specifically, the processor can compare the m cumulative displacement changes corresponding to m effective target feature points with a preset cumulative displacement change to obtain the number k of effective cumulative displacement changes that are greater than the preset cumulative displacement change. The processor then determines the ratio of the number k of effective cumulative displacement changes to the total number m of cumulative displacement changes, i.e., the second ratio, and compares the second ratio with the second preset ratio. If the second ratio is greater than the second preset ratio, the processor can determine that the support state of the outrigger is either outrigger collapse or outrigger elevation. If the second ratio is less than or equal to the second preset ratio, the processor can determine that the support state of the outrigger is normal support.

[0066] In one specific embodiment Figure 2 This schematic diagram illustrates the framework of a system for identifying the support state of outriggers according to an embodiment of the present invention. Figure 3 The schematic diagram illustrates an engineering device according to an embodiment of the present invention. Figure 4 The illustration shows a schematic diagram of the image coordinate system and the positional changes of target feature points in an embodiment of the present invention. For example... Figure 2 As shown, a system for identifying the outrigger support status may include a camera, a processor, and an alarm unit. The camera can be used to acquire video image sequences, the processor can be used to perform feature analysis and outrigger collapse identification on the image sequences, and the alarm unit is an outrigger collapse risk output device; alarm methods may include voice, warning lights, etc. The system for identifying the outrigger support status can be installed on engineering equipment with outriggers, which may include, but is not limited to, boom-type engineering vehicles such as concrete pump trucks and cranes equipped with working outriggers.

[0067] like Figure 3 As shown, the camera can be mounted horizontally on the outside of the outrigger. The outrigger and camera remain relatively stationary. When the outrigger is not collapsed, the video footage captured by the camera is static. When the outrigger collapses, the video footage captured by the camera moves synchronously. Figure 4 As shown, assuming the outrigger is slowly collapsing downwards, the camera mounted on the collapsing outrigger will move downwards synchronously. A fixed target feature point P (e.g., a fixed point on the wall) in the video sequence captured by the camera will move upwards from position (x1, y1) to position (x2, y2) in the image. In the image coordinate system, the position of the target feature point P changes between adjacent frames. The image coordinate system can be defined as follows: the horizontal axis to the right is the positive x-direction, and the vertical axis downwards is the positive y-direction. Understandably, the vertical axis direction is the height direction.

[0068] Specifically, the process for identifying the outrigger support status is as follows:

[0069] First, let's take the number of target feature points as an example.

[0070] (1) At time T1, the camera captures the first frame image. The processor extracts the feature points of the image and calculates the coordinates of the feature point P of the target to be tracked. The coordinates are (x1, y1). (x1, y1) is added to the array set G and assigned the values: x0 = x1, y0 = y1.

[0071] (2) At time T2, the camera captured the second frame image, and the processor calculated the updated coordinates (x2, y2) of the target feature point P in the same way.

[0072] (3) Calculate the displacement vector s = (x2-x0, y2-y0) of the target feature point P. When the displacement vector of P along the y-direction (i.e., the height direction) changes in displacement (i.e., the magnitude of the component along the y-direction) satisfies If the condition is met, (x2, y2) is added to the array set G (i.e., the effective point set) and assigned the value: x0 = x2, y0 = y2, and step (4) is executed, where λ1 is an image pixel threshold; otherwise, step (2) is executed.

[0073] (4) Repeat steps (2) and (3) to add the coordinates of n target feature points P that meet the threshold condition to the array set G.

[0074] (5) Calculate the displacement vector s1 = (x_i - x_i) between the first coordinate element and the nth coordinate element in the array set G. n -x1,y n -y1), if satisfying If the condition is met, it indicates that the pixel displacement of the target feature point P is caused by the movement of the camera, and the movement of the camera is due to the risk of collapse or lifting of its supporting leg. The processor then executes a warning output. Otherwise, the array set G is cleared, and step (1) is continued. This indicates the magnitude of the component of s1 along the y-direction.

[0075] The target points here are prominent feature points of the image, generally corner points, which are locations with large gray-level gradients in the image. Multiple corner points can typically be extracted from a single image frame. The method described above only uses a corner point as an example to illustrate the positional changes of the target point. In practical applications, the position of a corner point may be easily affected by its own vibration, external environmental disturbances (such as leaf movement), etc. To address external interference and improve the universality of the method, the number of target feature points can be multiple. When the number of target feature points is multiple, the above steps can be further optimized as follows:

[0076] (1) At time T1, the camera acquires the first frame of the video sequence. The processor calculates the feature points of the image and extracts m significant corner points from the first frame as the target feature point set to be tracked and analyzed. The coordinate set of the m target feature points is {(x... 11 ,y 11 ),(x 12 ,y 12 ...(x) 1m ,y 1m The initial set of m target feature points is stored in an empty array G, and {(x)} is initialized. 01 ,y 01 ),(x 02 ,y 02 ...(x) 0m ,y 0m )}={(x 11 ,y 11 ),(x 12 ,y 12 ...(x) 1m ,y 1m )};

[0077] (2) At time T2, the second frame image captured by the camera is processed, and the initial set of m target feature points is tracked and calculated to obtain the updated coordinate set corresponding to each target feature point as {(x 21 ,y 21 ),(x 22 ,y 22 ...(x) 2m ,y 2m )};

[0078] (3) Traverse each target feature point and calculate the updated displacement vector s of the i-th target feature point. i =(x 2i -x 0i ,y 2i -y 0i ), i∈[1,m], count l of the m target feature points whose displacement change along the y-direction is greater than the threshold λ1, if Then, the updated target feature point set is saved to the array set G, and the initial target feature point coordinates are updated to {(x... 01 ,y 01 ),(x 02 ,y 02 ...(x) 0m ,y 0m )}={(x 21 ,y 21 ),(x 22 ,y 22 ...(x)2m ,y 2m Otherwise, repeat step (2);

[0079] (4) Repeat steps (2) and (3) until n sets of target feature points are stored in the data set G;

[0080] (5) Calculate the displacement vector s of the corresponding points in the first coordinate set and the nth coordinate set in the array set G. j =(x nj -x 1j ,y nj -y 1j ), j∈[1,m], statistical calculation If the number is k, then This indicates that multiple target feature points have moved a certain amount along the same y-direction, suggesting that the camera has moved due to the collapse or lifting of the outriggers. Otherwise, clear the array set G and continue to re-initialize the target feature point set according to step (1) for analysis. Let y be the displacement component of the j-th target feature point in the target feature point set along the y direction.

[0081] When there are multiple target feature points, the above method can, to some extent, solve the problem of periodic vibration of the image target caused by camera vibration and background target disturbance. The specific explanation is as follows:

[0082] (1) First, extract m target feature points as the target feature point set for tracking analysis, which can avoid misjudging the global target motion due to external interference of individual points;

[0083] (2) If the target feature point sets of two adjacent frames satisfy the following conditions This indicates that the target feature point set has a collective and consistent movement, in which case invalid and chaotic interference movements can be eliminated;

[0084] (3) After accumulating a certain amount of exercise, if This indicates that the cumulative motion of the target feature point set along the same direction exceeds the warning threshold. At this point, only valid motion types that move continuously in one direction match the motion model characteristics of the camera caused by the outrigger collapse. Therefore, interfering periodic motion types can be excluded.

[0085] In summary, the technical solution of this invention transforms the identification of outrigger collapse in engineering equipment into a problem of tracking target feature points in a video image sequence. It predicts the outrigger collapse state by establishing a motion model of the target feature points in the image. The outrigger collapse identification problem is transformed into a camera motion detection problem, which involves tracking target feature points in the scene and calculating the cumulative displacement of these feature points for identification. Specifically, during a tracking and analysis period, m corner points are first extracted from the first frame image as target feature points to be tracked. Then, optical flow is used to track these target feature points. If a feature point experiences a certain displacement along the y-direction (>λ1), it indicates that the displacement of the feature point is effective; otherwise, it indicates that the feature point is stationary. If l feature points experience effective displacement and... This indicates that the feature points exhibit collective movement (for example, if 100 feature points are extracted, and 80% of them undergo effective displacement, it indicates collective movement of the feature points, and conversely, it suggests that the camera has moved effectively; however, the specific type of movement includes periodic movement and linear movement). Finally, these m feature points are continuously tracked, and the coordinates of the feature points under n sets of effective displacements are recorded. The displacements along the y-direction of the first and last sets of feature points are calculated. If a certain proportion of the feature points satisfy... This indicates that the tracked feature point has undergone effective displacement along the y-direction, and the movement is linear rather than periodic. In other words, the cumulative displacement of the periodic movement is too small to meet the set parameters. This eliminates periodic image movement caused by mechanical vibration and external interference, improving the accuracy of outrigger collapse recognition.

[0086] Furthermore, the technical solution provided by the embodiments of the present invention can solve the problem of periodic interference caused by the small vibrations of the machine itself and the background disturbances of the external environment (disturbances in the local target area) by tracking one or more target feature points according to the set tracking strategy.

[0087] This invention provides a processor configured to execute the method for identifying the support state of a leg according to the above embodiments.

[0088] This invention provides a device 500 for identifying the support state of outriggers, applied to engineering equipment including outriggers. An image acquisition device is mounted on the outrigger for acquiring image sequences. The device includes:

[0089] The location acquisition module 510 is used to acquire the location information of target feature points in the image sequence.

[0090] The effective point set determination module 520 is used to determine the effective point set based on the position information of the target feature points. The effective point set is a collection of effective target feature points among the target feature points, and the displacement change along the height direction between adjacent effective target feature points reaches a preset displacement change amount.

[0091] The cumulative displacement determination module 530 is used to determine the cumulative displacement change of the effective target feature points along the height direction based on the position information of the effective target feature points.

[0092] The support status determination module 540 is used to determine the support status of the outriggers based on the cumulative displacement change.

[0093] The above technical solution involves setting up an image acquisition device on the outrigger and acquiring the position information of target feature points in the image sequence. Based on the position information of the target feature points, a set of effective points is determined, where the set of effective target feature points is the collection of effective target feature points. The displacement change along the height direction between adjacent effective target feature points reaches a preset displacement change amount. Then, based on the position information of the effective target feature points, the cumulative displacement change of the effective target feature points along the height direction is determined, thereby determining the support state of the outrigger based on the cumulative displacement change amount. The above technical solution eliminates the need for operators to visually identify the support status of the outriggers, reducing reliance on operators and improving equipment safety. Based on the positional information of target feature points, effective target feature points that have undergone effective movement in the height direction are selected. The presence of group movement among these effective feature points can be determined by analyzing their cumulative displacement along the height direction. Since the image acquisition device is mounted on the outriggers, the support status of the outriggers can be determined based on the overall movement direction of the effective target feature points in the height direction within the image sequence. This achieves accurate identification of the outrigger support status and improves the accuracy of outrigger support status identification.

[0094] In one embodiment, the support state determination module 540 is further configured to: compare the cumulative displacement change with a preset cumulative displacement change; if the cumulative displacement change is greater than the preset cumulative displacement change, determine that the support state of the outrigger is outrigger collapse or outrigger lifting; if the cumulative displacement change is not greater than the preset cumulative displacement change, determine that the support state of the outrigger is normal outrigger support.

[0095] In one embodiment, the number of valid target feature points in the valid point set is no greater than a preset upper limit threshold.

[0096] In one embodiment, the location acquisition module 510 is further configured to: acquire the location information of target feature points in an image sequence within a preset time period.

[0097] In one embodiment, the number of target feature points is m, where m is greater than 1; the effective point set determination module 520 is further configured to: determine the displacement change of the m target feature points in the current frame image relative to the corresponding m target feature points in the previous effective frame image along the height direction based on the position information of the target feature points, so as to obtain m displacement changes, wherein the effective image is an image in the image sequence in which all m target feature points are located in the effective point set; determine the number h of effective displacement changes that reach a preset displacement change among the m displacement changes; determine the ratio of the number h of effective displacement changes to the number m of displacement changes, so as to obtain a first ratio; if the first ratio is determined to be greater than the first preset ratio, add the m target feature points in the current frame image to the effective point set; if the first ratio is determined to be less than or equal to the first preset ratio, do not add the m target feature points in the current frame image to the effective point set.

[0098] In one embodiment, the number of target feature points is single; the cumulative displacement determination module 530 is further configured to: determine the displacement change along the height direction between the last valid target feature point in the set of valid points and the first valid target feature point in the set of valid points, so as to obtain the cumulative displacement change.

[0099] In one embodiment, the number of target feature points is m, where m is greater than 1; the cumulative displacement determination module 530 is further used to: determine the displacement changes along the height direction of the m effective target feature points in the last group of the effective point set and the m effective target feature points in the first group of the effective point set, so as to obtain m cumulative displacement changes.

[0100] In one embodiment, the support state determination module 540 is further configured to: determine the number k of effective cumulative displacement changes greater than a preset cumulative displacement change among m cumulative displacement changes; determine the ratio of the number k of effective cumulative displacement changes to the number m of cumulative displacement changes to obtain a second ratio; if the second ratio is greater than the second preset ratio, determine the support state of the outrigger as outrigger collapse or outrigger lifting; if the second ratio is less than or equal to the second preset ratio, determine the support state of the outrigger as normal outrigger support.

[0101] This invention provides an engineering device, including: a support leg; an image acquisition device disposed on the support leg for acquiring image sequences; and a processor according to the above embodiments or a device according to the above embodiments for identifying the support state of the support leg.

[0102] This invention provides a machine-readable storage medium storing a program or instructions that, when executed by a processor, implement the method for identifying the support state of a leg according to the above embodiments.

[0103] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0105] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0106] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0107] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0108] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0109] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0110] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0111] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for identifying the support state of outriggers, applied to engineering equipment including outriggers, characterized in that, An image acquisition device is installed on the outrigger, and the image acquisition device is used to acquire image sequences. The image acquisition device is installed horizontally on the outside of the outrigger, and the outrigger and the image acquisition device remain relatively stationary. When the outrigger does not collapse, the image sequence acquired by the image acquisition device is in a static state. When the outrigger collapses, the image sequence acquired by the image acquisition device moves synchronously. The method includes: Obtain the location information of the target feature points in the image sequence; A valid point set is determined based on the location information of the target feature points, wherein the valid point set is a collection of valid target feature points among the target feature points, and the displacement change along the height direction between adjacent valid target feature points reaches a preset displacement change amount; The cumulative displacement change of the effective target feature point along the height direction is determined based on the position information of the effective target feature point; The support state of the outrigger is determined based on the cumulative displacement change.

2. The method according to claim 1, characterized in that, Determining the support state of the outrigger based on the cumulative displacement change includes: Compare the cumulative displacement change with a preset cumulative displacement change; If the cumulative displacement change is greater than the preset cumulative displacement change, the support state of the outrigger is determined to be either outrigger collapse or outrigger lifting. If the cumulative displacement change is not greater than the preset cumulative displacement change, the support state of the outrigger is determined to be normal support.

3. The method according to claim 1, characterized in that, The number of valid target feature points in the set of valid points is not greater than a preset upper limit threshold.

4. The method according to claim 1, characterized in that, The step of obtaining the location information of the target feature points in the image sequence includes: Obtain the location information of target feature points in the image sequence within a preset time period.

5. The method according to claim 1, characterized in that, The number of target feature points is m, where m is greater than 1; determining the effective point set based on the location information of the target feature points includes: Based on the position information of the target feature points, determine the displacement changes of m target feature points in the current frame image relative to the corresponding m target feature points in the previous valid image along the height direction, so as to obtain m displacement changes. The valid image is an image in the image sequence in which all m target feature points are located in the set of valid points. Determine the number h of effective displacement changes that reach the preset displacement change among the m displacement changes; Determine the ratio of the number of effective displacement changes h to the number of displacement changes m to obtain a first ratio; If it is determined that the first ratio is greater than the first preset ratio, the m target feature points in the current frame image are added to the effective point set; If the first ratio is determined to be less than or equal to the first preset ratio, the m target feature points in the current frame image are not added to the effective point set.

6. The method according to claim 1, characterized in that, The number of target feature points is single; determining the cumulative displacement change of the effective target feature points along the height direction based on the position information of the effective target feature points includes: The displacement change along the height direction between the last effective target feature point in the effective point set and the first effective target feature point in the effective point set is determined to obtain the cumulative displacement change.

7. The method according to claim 1, characterized in that, The number of target feature points is m, where m is greater than 1; The step of determining the cumulative displacement change of the effective target feature point along the height direction based on the position information of the effective target feature point includes: Determine the displacement changes along the height direction of the m effective target feature points in the last group of the effective point set and the m effective target feature points in the first group of the effective point set, respectively, to obtain m cumulative displacement changes.

8. The method according to claim 7, characterized in that, Determining the support state of the outrigger based on the cumulative displacement change includes: Determine the number k of effective cumulative displacement changes that are greater than a preset cumulative displacement change among the m cumulative displacement changes; Determine the ratio of the number of effective cumulative displacement changes k to the number of cumulative displacement changes m to obtain a second ratio; If the second ratio is greater than the second preset ratio, the support state of the outrigger is determined to be either outrigger collapse or outrigger elevation. If the second ratio is determined to be less than or equal to the second preset ratio, the support state of the outrigger is determined to be normal support.

9. A processor, characterized in that, It is configured to perform the method for identifying the support state of the outrigger as described in any one of claims 1 to 8.

10. A device for identifying the support state of outriggers, applied to engineering equipment including outriggers, characterized in that, An image acquisition device is installed on the outrigger. This device is used to acquire image sequences and is horizontally mounted on the outside of the outrigger. The outrigger and the image acquisition device remain relatively stationary. When the outrigger does not collapse, the image sequence acquired by the image acquisition device is static. When the outrigger collapses, the image sequence acquired by the image acquisition device moves synchronously. The device includes: The location acquisition module is used to acquire the location information of target feature points in the image sequence; The effective point set determination module is used to determine the effective point set based on the position information of the target feature points, wherein the effective point set is a collection of effective target feature points among the target feature points, and the displacement change along the height direction between adjacent effective target feature points reaches a preset displacement change amount; The cumulative displacement determination module is used to determine the cumulative displacement change of the effective target feature point along the height direction based on the position information of the effective target feature point; The support state determination module is used to determine the support state of the outrigger based on the cumulative displacement change.

11. An engineering device, characterized in that, include: Support legs; An image acquisition device is mounted on the support leg and is used to acquire image sequences. The image acquisition device is installed horizontally on the outside of the support leg. The support leg and the image acquisition device remain relatively stationary. When the support leg does not collapse, the image sequence acquired by the image acquisition device is in a static state. When the support leg collapses, the image sequence acquired by the image acquisition device moves synchronously. as well as The processor according to claim 9 or the device for identifying the support state of the outrigger according to claim 10.

12. A machine-readable storage medium on which a program or instructions are stored, characterized in that, When the program or the instructions are executed by the processor, they implement the method for identifying the support state of the outriggers according to any one of claims 1 to 8.

Images