Anti-micro-jitter deformation and displacement monitoring method and device based on machine vision

Abstract

Embodiments of the present application disclose a method and device for anti-micro-jitter deformation and displacement monitoring based on machine vision. The method comprises: acquiring an image captured by a camera; performing target detection on the image to obtain an initial target position and multi-scale features; determining a fitting result according to the initial target position and a regression result according to the multi-scale features; judging whether the target of the image is lost according to the fitting result and the regression result; if the target of the image is not lost, performing sub-pixel fitting according to the fitting result and the regression result to obtain a real physical deformation variable of the monitoring target. Through the method of the embodiments of the present application, the target can be quickly found back when the camera is jittered, and the deformation of the target can be continuously detected, so that the accuracy and stability of deformation monitoring are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to a machine vision algorithm, more particularly to a method and device for anti-micro-jitter deformation and displacement monitoring based on machine vision. BACKGROUND

[0002] With the development of machine vision algorithm technology, more and more researchers apply it to safety monitoring, especially in the direction of non-contact deformation monitoring based on machine vision. The commonly used monitoring technology at present is the feature point matching or template matching algorithm based on template, i.e. natural target or artificial target, and its technical process is as follows: manually calibrating the target; calculating the current scale factor; performing feature point matching / template matching to calculate the pixel deformation variable; and converting it into the actual deformation variable

[0003] The existing monitoring technology often needs to be based on the following assumptions: the camera focal length does not change during operation; and the camera does not move. However, in actual outdoor monitoring scenarios, the above assumptions are often unachievable, especially the assumption that the camera does not move. Therefore, the deformation monitoring technology based on manual calibration has the problems of small monitoring field of view, only monitoring the deformation in the calibrated ROI region, the target is likely to be lost if the camera shakes or sinks, no automatic calibration function, and insufficient intelligence.

[0004] Therefore, it is necessary to design a new method to quickly find the target when the camera shakes and continue to detect the deformation of the target, thereby improving the accuracy and stability of deformation monitoring. SUMMARY

[0005] The present application aims to overcome the defects of the prior art and provide a method and device for anti-micro-jitter deformation and displacement monitoring based on machine vision.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solution: a method for anti-micro-jitter deformation and displacement monitoring based on machine vision, comprising:

[0007] obtaining an image captured by a camera;

[0008] performing target detection on the image to obtain an initial target position and multi-scale features;

[0009] determining a fitting result according to the initial target position and a regression result according to the multi-scale features;

[0010] judging whether the target of the image is lost according to the fitting result and the regression result;

[0011] if the target of the image is not lost, performing sub-pixel fitting according to the fitting result and the regression result to obtain the actual physical deformation variable of the monitoring target.

[0012] A further technical solution is that the image is subjected to target detection to obtain a target initial position and multi-scale features, including:

[0013] The image is subjected to target detection by using a YOLOv7 model to obtain a target initial position and multi-scale features.

[0014] A further technical solution is that the fitting result is determined according to the target initial position, and the regression result is determined according to the multi-scale features, including:

[0015] The multi-scale features are subjected to position regression to obtain a regression result.

[0016] The target initial position is subjected to geometric center fitting, and a scale factor is calculated to obtain a fitting result.

[0017] A further technical solution is that the multi-scale features are subjected to position regression to obtain a regression result, including:

[0018] The detection frame with a confidence greater than a set value in the target frame subjected to regression in the multi-scale features is subjected to non-maximum threshold processing to obtain three optimal detection frames.

[0019] The average value of the three optimal detection frames is calculated to obtain a regression result.

[0020] A further technical solution is that the real physical deformation variable of the monitoring target is obtained by performing sub-pixel fitting according to the fitting result and the regression result, including:

[0021] The fitting result and the regression result are subjected to sub-pixel aggregation to obtain a final pixel deformation value.

[0022] The final pixel deformation value is used for scale conversion to obtain the real physical deformation variable of the monitoring target.

[0023] A further technical solution is that the fitting result and the regression result are subjected to sub-pixel aggregation to obtain a final pixel deformation value, including:

[0024] The pixel deformation variable of the fitting result and the pixel deformation variable of the regression result are calculated.

[0025] The average value of the pixel deformation variable of the fitting result and the pixel deformation variable of the regression result is calculated to obtain a final pixel deformation value.

[0026] A further technical solution is that the final pixel deformation value is used for scale conversion to obtain the real physical deformation variable of the monitoring target, including:

[0027] According to the final pixel deformation value multiplied by the scale factor, a real physical deformation variable of a monitoring target is obtained.

[0028] The application further provides a machine vision-based anti-micro-jitter deformation and displacement monitoring device, comprising:

[0029] An image acquisition unit is configured to acquire an image captured by a camera.

[0030] A target detection unit is configured to perform target detection on the image to obtain a target initial position and multi-scale features.

[0031] A result determination unit is configured to determine a fitting result according to the target initial position and determine a regression result according to the multi-scale features.

[0032] A judgment unit is configured to judge whether the target of the image is lost according to the fitting result and the regression result.

[0033] A sub-pixel fitting unit is configured to perform sub-pixel fitting according to the fitting result and the regression result to obtain a real physical deformation variable of a monitoring target if the target of the image is not lost.

[0034] The application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the method described above when executing the computer program.

[0035] The application further provides a storage medium storing a computer program, wherein the computer program is executed by a processor to implement the method described above.

[0036] Compared with the prior art, the application has the following beneficial effects: the application performs target detection on an image, geometric center fitting on a target initial position, position regression on multi-scale features, and average calculation on matching results and geometric center fitting at different scales of a feature pyramid to achieve a sub-pixel fitting effect, so that the target can be quickly found back when a camera is jittered, and the deformation of the target can be continuously detected, thereby improving the precision and stability of deformation monitoring.

[0037] The application will be further described below in combination with the drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0039] Figure 1 An application scenario diagram of the anti-micro-jitter deformation and displacement monitoring method based on machine vision provided by the embodiment of the present application is shown in FIG. 1.

[0040] Figure 2 A flow diagram of the anti-micro-jitter deformation and displacement monitoring method based on machine vision provided by the embodiment of the present application is shown in FIG. 2.

[0041] Figure 3 A sub-flow diagram of the anti-micro-jitter deformation and displacement monitoring method based on machine vision provided by the embodiment of the present application is shown in FIG. 3.

[0042] Figure 4 A sub-flow diagram of the anti-micro-jitter deformation and displacement monitoring method based on machine vision provided by the embodiment of the present application is shown in FIG. 4.

[0043] Figure 5 A sub-flow diagram of the anti-micro-jitter deformation and displacement monitoring method based on machine vision provided by the embodiment of the present application is shown in FIG. 5.

[0044] Figure 6 A sub-flow diagram of the anti-micro-jitter deformation and displacement monitoring method based on machine vision provided by the embodiment of the present application is shown in FIG. 6.

[0045] Figure 7 A schematic block diagram of the anti-micro-jitter deformation and displacement monitoring device based on machine vision provided by the embodiment of the present application is shown in FIG. 7.

[0046] Figure 8 A schematic block diagram of the computer device provided by the embodiment of the present application is shown in FIG. 8.

[0047] Figure 9 A schematic diagram of the anti-micro-jitter deformation and displacement monitoring example based on machine vision provided by the embodiment of the present application is shown in FIG. 9. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0049] It should be understood that, when used in the present specification and the appended claims, the terms "comprise" and "include" indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0050] It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in this specification and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0051] It should be further understood that the term "and / or" used in the specification and the appended claims, means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0052] Please refer to Figure 1 and Figure 2 , Figure 1 The application scenario schematic diagram of the anti-micro-jitter deformation and displacement monitoring method based on machine vision provided by the embodiment of the present application. Figure 2 The schematic flowchart of the anti-micro-jitter deformation and displacement monitoring method based on machine vision provided by the embodiment of the present application. The anti-micro-jitter deformation and displacement monitoring method based on machine vision is applied to a server. The server interacts with a camera to quickly find a target when the camera is jittered and continue to detect the deformation of the target, thereby improving the accuracy and stability of deformation monitoring. It can be trained end-to-end and is very efficient, with a speed of up to 150 FPS. When the monitoring device is jittered or offset, it can quickly find the monitoring target and continue to monitor. It combines deep learning algorithm technology and traditional image processing algorithm technology, making the entire monitoring system more stable and easy to migrate.

[0053] Figure 2 The flowchart of the anti-micro-jitter deformation and displacement monitoring method based on machine vision provided by the embodiment of the present application. As shown in Figure 2 , the method comprises the following steps S110 to S150.

[0054] S110, acquiring an image photographed by a camera.

[0055] In this embodiment, the image refers to a visual image photographed by a camera.

[0056] S120, performing target detection on the image to obtain a target initial position and multi-scale features.

[0057] In this embodiment, the target initial position refers to the position information of the target after target recognition of the image; the multi-scale features refer to three different scale feature maps output by a target detection model.

[0058] Specifically, a YOLOv7 model is used to perform target detection on the image to obtain the target initial position and the multi-scale features.

[0059] In the embodiment, the target in the image is labeled by the labelImg labeling software to obtain a sample picture; a YOLOv7 algorithm model is built based on a Pytorch deep learning framework; and the target image is detected by using the YOLOv7 algorithm model.

[0060] Specifically, a target image with a size of 640x640 is input into the YOLOv7 model, and feature maps with relative original Figure 1 sizes of 1 / 8, 1 / 16 and 1 / 32 and an initial position of the target are sequentially obtained.

[0061] S130, determining a fitting result according to the initial position of the target and a regression result according to the multi-scale features.

[0062] In the embodiment, the fitting result refers to a result of geometric center fitting of the initial position of the target; and the regression result refers to a result of position regression of the multi-scale features.

[0063] In an embodiment, referring to Figure 3 , the step S130 can include steps S131-S132.

[0064] S131, performing position regression on the multi-scale features to obtain the regression result.

[0065] In an embodiment, referring to Figure 4 , the step S131 can include steps S1311-S1312.

[0066] S1311, performing non-maximum threshold processing on a detection frame with a confidence greater than a set value in a target frame regressed from the multi-scale features to obtain three optimal detection frames.

[0067] S1312, calculating an average value of the three optimal detection frames to obtain the regression result.

[0068] Specifically, the upper left coordinates and the lower right coordinates of the optimal detection frames obtained from the three scales (1 / 8, 1 / 16 and 1 / 32) features are denoted as a vector V i ; V i =(x1,y1,x2,y1)i={1 / 8,1 / 16,1 / 32};

[0069] Specifically, the detection frames with a confidence greater than 0.6 in the target frames regressed from the three different scale feature maps (1 / 8, 1 / 16 and 1 / 32) are subjected to non-maximum threshold (NMS) processing, the optimal detection frames are obtained respectively, and the three optimal detection frames are averaged to achieve sub-pixel effect.

[0070] S132, geometric center fitting is performed on the target initial position, and a scale factor is calculated to obtain a fitting result.

[0071] In this embodiment, geometric center fitting is performed on a circular pattern in a final target frame output by the detection model, and a scale factor is calculated.

[0072] S140, whether the target of the image is lost is determined according to the fitting result and the regression result.

[0073] Whether the target is lost is mainly determined in two ways. First, the target detection branch has two outputs, one is the sub-pixel position information of the monitored target under three different scales, and the initial position of the target (ROI region) determined based on the same. After obtaining the initial position, the system locally stores an ROI image (init_roi_img) of the target. When a new picture is obtained, the hash similarity hash_value of the target image corresponding to the current picture and init_roi_img is calculated. Moreover, the system performs ellipse fitting in the ROI region determined by the target detection to obtain the center coordinates of the ellipse, the length of the major axis and the minor axis, a and b. The fitting coefficient E = a / b; S = (hash_value + E) / 2. When S is greater than or equal to 0.9, it indicates that the target is not lost, otherwise it indicates that the target is lost.

[0074] S150, if the target of the image is not lost, sub-pixel fitting is performed according to the fitting result and the regression result to obtain the real physical deformation variable of the monitored target.

[0075] In this embodiment, the real physical deformation variable of the monitored target refers to the real physical deformation variable of the target, i.e., the target.

[0076] In an embodiment, referring to Figure 5 The above step S150 can include steps S151-S152.

[0077] S151, sub-pixel aggregation is performed according to the fitting result and the regression result to obtain a final pixel deformation value.

[0078] In this embodiment, the final pixel deformation value refers to the result of sub-pixel aggregation of the fitting result and the regression result.

[0079] In an embodiment, referring to Figure 6 The above step S151 can include steps S1511-S1512.

[0080] S1511, the pixel deformation variable of the fitting result and the pixel deformation variable of the regression result are calculated.

[0081] The calculation of the pixel deformation variable is divided into two parts, one part is based on the regression result of target detection, and the other part is based on the center coordinate result of the ellipse fitting of the ROI region. First, when the system is started, initialization is performed to obtain the following initial quantities of the monitored target: the initial center coordinate (init_center), and the coordinates of the minimum circumscribed rectangle of the target on the left top and the right bottom. When monitoring is performed, the multi-scale target contour coordinates output by the target detection algorithm module and the center coordinates obtained by the ellipse fitting are subtracted from the initial target circumscribed rectangle coordinates and the initial center coordinates, respectively, to obtain the pixel deformation variables corresponding to the two.

[0082] S1512, calculate the average of the pixel deformation variable of the fitting result and the pixel deformation variable of the regression result to obtain the final pixel deformation value.

[0083] S152, perform scale conversion using the final pixel deformation value to obtain the real physical deformation variable of the monitored target.

[0084] Specifically, the final pixel deformation value is multiplied by the scale factor to obtain the real physical deformation variable of the monitored target.

[0085] In the embodiment, the pixel deformation variables of the fitting result and the regression result are calculated respectively, and the average of the two is taken as the final pixel deformation value; the final pixel deformation value is multiplied by the scale factor to obtain the real physical deformation variable of the monitored target.

[0086] If the target of the image is lost, the step S120 is performed.

[0087] The method of the embodiment mainly realizes micro-deformation monitoring and target tracking through target detection + geometric figure fitting + image similarity calculation. The target tracking refers to that when the target jumps out of the search region (ROI region) due to camera shaking, the target detection module can make a judgment in combination with the fitting coefficient and the hash similarity evaluation to update the ROI region, thereby achieving the effect of not losing the target.

[0088] For example, as shown in Figure 9 When the camera does not occur large shaking (initial state), according to the calibrated ROI region, the algorithm performs target detection on the whole image and ellipse fitting in the ROI region to obtain the center coordinates. When the camera shakes and causes the monitored target to jump out of the ROI region, causing the target to be lost, the system triggers the target detection module to obtain the latest position of the target, and performs frame expansion according to the detection frame (the expansion scale is 1 / 3 of the length w of the detection frame). The obtained expanded frame updates the original ROI region frame. As shown in Figure 9As shown in the "shaking offset state", the rectangular box is the updated ROI region, the circle offset from the camera lens is the position of the original target circle, the circle corresponding to the camera lens is the circle fitted in the current frame, and the small rectangular box is the target detection box of the current target frame.

[0089] The anti-micro-shaking deformation and displacement monitoring method based on machine vision has the advantages that the image is subjected to target detection, the geometric center fitting is performed on the initial position of the target, the position regression is performed on the multi-scale features, the matching results under different scales of the feature pyramid and the geometric center fitting are averaged to achieve the effect of sub-pixel fitting, the target can be quickly found back when the camera shakes, the deformation of the target can be continuously detected, and the accuracy and stability of the deformation monitoring are improved.

[0090] Figure 7 The anti-micro-shaking deformation and displacement monitoring device 300 based on machine vision provided by the embodiment of the application is a schematic block diagram. As shown in Figure 7 According to the anti-micro-shaking deformation and displacement monitoring method based on machine vision, the application further provides an anti-micro-shaking deformation and displacement monitoring device 300 based on machine vision. The anti-micro-shaking deformation and displacement monitoring device 300 based on machine vision comprises units for executing the anti-micro-shaking deformation and displacement monitoring method based on machine vision, and the device can be configured in a server. Specifically, please refer to Figure 7 The anti-micro-shaking deformation and displacement monitoring device 300 based on machine vision comprises an image acquisition unit 301, a target detection unit 302, a result determination unit 303, a judgment unit 304, and a sub-pixel fitting unit 305.

[0091] The image acquisition unit 301 is configured to acquire an image captured by a camera; the target detection unit 302 is configured to perform target detection on the image to obtain an initial position of a target and multi-scale features; the result determination unit 303 is configured to determine a fitting result according to the initial position of the target and determine a regression result according to the multi-scale features; the judgment unit 304 is configured to judge whether the target of the image is lost according to the fitting result and the regression result; and the sub-pixel fitting unit 305 is configured to perform sub-pixel fitting according to the fitting result and the regression result to obtain a real physical deformation variable of a monitoring target if the target of the image is not lost.

[0092] In an embodiment, the target detection unit 302 is configured to perform target detection on the image by using a YOLOv7 model to obtain the initial position of the target and the multi-scale features.

[0093] In an embodiment, the result determination unit 303 comprises a position regression sub-unit and a geometric fitting sub-unit.

[0094] a position regression subunit configured to perform position regression on the multi-scale features to obtain a regression result; and a geometric fitting subunit configured to perform geometric center fitting on the target initial position and calculate a scale factor to obtain a fitting result.

[0095] In an embodiment, the position regression subunit includes a processing module and a mean value calculation module.

[0096] The processing module is configured to perform non-maximum threshold processing on the bounding boxes in the regression target frame in the multi-scale features whose confidence is greater than a set value to obtain three optimal bounding boxes; and the mean value calculation module is configured to calculate the mean value of the three optimal bounding boxes to obtain the regression result.

[0097] In an embodiment, the subpixel fitting unit 305 includes an aggregation subunit and a scale conversion subunit.

[0098] The aggregation subunit is configured to perform subpixel aggregation on the fitting result and the regression result to obtain a final pixel deformation value; and the scale conversion subunit is configured to perform scale conversion using the final pixel deformation value to obtain a real physical deformation variable of the monitoring target.

[0099] In an embodiment, the aggregation subunit includes a deformation variable calculation module and a final deformation value calculation module.

[0100] The deformation variable calculation module is configured to calculate the pixel deformation variable of the fitting result and the pixel deformation variable of the regression result; and the final deformation value calculation module is configured to calculate the mean value of the pixel deformation variable of the fitting result and the pixel deformation variable of the regression result to obtain the final pixel deformation value.

[0101] In an embodiment, the scale conversion subunit is configured to multiply the final pixel deformation value by the scale factor to obtain the real physical deformation variable of the monitoring target.

[0102] It should be noted that the specific implementation process of the above-mentioned machine vision-based anti-micro-jitter deformation and displacement monitoring device 300 and each unit can be clearly understood by those skilled in the art, which can be referred to the corresponding description in the foregoing method embodiments. For the convenience and brevity of description, it will not be repeated here.

[0103] The above-mentioned machine vision-based anti-micro-jitter deformation and displacement monitoring device 300 can be realized in the form of a computer program, which can run on a computer device as shown in Figure 8 .

[0104] Please refer to Figure 8 , Figure 8is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 500 can be a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0105] Referring to Figure 8 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501, wherein the memory can include a non-volatile storage medium 503 and an internal memory 504.

[0106] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, can cause the processor 502 to perform a machine vision-based anti-micro-jitter deformation and displacement monitoring method.

[0107] The processor 502 is configured to provide computing and control capabilities to support the operation of the entire computer device 500.

[0108] The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503, which, when executed by the processor 502, can cause the processor 502 to perform a machine vision-based anti-micro-jitter deformation and displacement monitoring method.

[0109] The network interface 505 is configured to perform network communication with other devices. Those skilled in the art can understand that Figure 8 The structure shown in FIG. 5 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device 500 to which the scheme of the present application is applied. Specifically, the computer device 500 can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0110] The processor 502 is configured to run the computer program 5032 stored in the memory to implement the following steps:

[0111] obtain an image captured by a camera; perform target detection on the image to obtain an initial target position and multi-scale features; determine a fitting result according to the initial target position and a regression result according to the multi-scale features; determine whether the target of the image is lost according to the fitting result and the regression result; if the target of the image is not lost, perform sub-pixel fitting according to the fitting result and the regression result to obtain a real physical deformation variable of the monitoring target.

[0112] In an embodiment, the processor 502, when implementing the step of performing target detection on the image to obtain a target initial position and multi-scale features, specifically implements the following steps:

[0113] The YOLOv7 model is used to perform target detection on the image to obtain a target initial position and multi-scale features.

[0114] In an embodiment, the processor 502, when implementing the step of determining a fitting result according to the target initial position and determining a regression result according to the multi-scale features, specifically implements the following steps:

[0115] The multi-scale features are subjected to position regression to obtain a regression result, and the target initial position is subjected to geometric center fitting and calculation of a scale factor to obtain a fitting result.

[0116] In an embodiment, the processor 502, when implementing the step of performing position regression on the multi-scale features to obtain a regression result, specifically implements the following steps:

[0117] The detection boxes with a confidence greater than a set value in the regression target boxes in the multi-scale features are subjected to non-maximum threshold processing to obtain three optimal detection boxes, and the average of the three optimal detection boxes is calculated to obtain a regression result.

[0118] In an embodiment, the processor 502, when implementing the step of performing sub-pixel fitting according to the fitting result and the regression result to obtain a real physical deformation variable of the monitoring target, specifically implements the following steps:

[0119] Sub-pixel aggregation is performed according to the fitting result and the regression result to obtain a final pixel deformation value, and scale conversion is performed using the final pixel deformation value to obtain a real physical deformation variable of the monitoring target.

[0120] In an embodiment, the processor 502, when implementing the step of performing sub-pixel aggregation according to the fitting result and the regression result to obtain a final pixel deformation value, specifically implements the following steps:

[0121] The pixel deformation variable of the fitting result and the pixel deformation variable of the regression result are calculated, and the average of the pixel deformation variable of the fitting result and the pixel deformation variable of the regression result is calculated to obtain a final pixel deformation value.

[0122] In an embodiment, the processor 502, when implementing the step of performing scale conversion using the final pixel deformation value to obtain a real physical deformation variable of the monitoring target, specifically implements the following steps:

[0123] The final pixel deformation value is multiplied by the scale factor to obtain a real physical deformation variable of the monitoring target.

[0124] It should be understood that, in the embodiments of the present application, the processor 502 can be a central processing unit (CPU), and the processor 502 can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0125] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the above-mentioned embodiments.

[0126] Therefore, the present application also provides a storage medium. The storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program is executed by a processor to make the processor perform the following steps:

[0127] obtaining an image captured by a camera; performing target detection on the image to obtain a target initial position and a multi-scale feature; determining a fitting result according to the target initial position and determining a regression result according to the multi-scale feature; judging whether the target of the image is lost according to the fitting result and the regression result; if the target of the image is not lost, performing sub-pixel fitting according to the fitting result and the regression result to obtain a real physical deformation variable of a monitoring target.

[0128] In an embodiment, when the processor executes the computer program to implement the step of performing target detection on the image to obtain a target initial position and a multi-scale feature, the processor specifically implements the following steps:

[0129] performing target detection on the image by using a YOLOv7 model to obtain a target initial position and a multi-scale feature.

[0130] In an embodiment, when the processor executes the computer program to implement the step of determining a fitting result according to the initial position of the target, and determining a regression result according to the multi-scale feature, the processor implements the following steps:

[0131] performing position regression on the multi-scale feature to obtain a regression result, and performing geometric center fitting on the initial position of the target and calculating a scale factor to obtain a fitting result.

[0132] In an embodiment, when the processor executes the computer program to implement the step of performing position regression on the multi-scale feature to obtain a regression result, the processor implements the following steps:

[0133] performing non-maximum thresholding on the bounding boxes in the regression target frame whose confidence is greater than a set value to obtain three optimal bounding boxes, and calculating the average of the three optimal bounding boxes to obtain the regression result.

[0134] In an embodiment, when the processor executes the computer program to implement the step of performing sub-pixel fitting according to the fitting result and the regression result to obtain a real physical deformation variable of the monitoring target, the processor implements the following steps:

[0135] performing sub-pixel aggregation according to the fitting result and the regression result to obtain a final pixel deformation value, and performing scale conversion using the final pixel deformation value to obtain a real physical deformation variable of the monitoring target.

[0136] In an embodiment, when the processor executes the computer program to implement the step of performing sub-pixel aggregation according to the fitting result and the regression result to obtain a final pixel deformation value, the processor implements the following steps:

[0137] calculating a pixel deformation variable of the fitting result and a pixel deformation variable of the regression result, and calculating the average of the pixel deformation variable of the fitting result and the pixel deformation variable of the regression result to obtain a final pixel deformation value.

[0138] In an embodiment, when the processor executes the computer program to implement the step of performing scale conversion using the final pixel deformation value to obtain a real physical deformation variable of the monitoring target, the processor implements the following steps:

[0139] multiplying the final pixel deformation value by the scale factor to obtain a real physical deformation variable of the monitoring target.

[0140] The storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, and various computer readable storage media that can store program codes.

[0141] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description in a general manner. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0142] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of each unit is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed.

[0143] The steps in the method embodiments of the present application can be adjusted, combined and deleted in sequence according to actual needs. The units in the device embodiments of the present application can be combined, divided and deleted according to actual needs. In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0144] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a terminal or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application.

[0145] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A machine vision-based method for monitoring micro-vibration deformation and displacement, characterized in that, include: Acquire images captured by the camera; Target detection is performed on the image to obtain the initial target position and multi-scale features; The fitting result is determined based on the initial position of the target, and the regression result is determined based on the multi-scale features; Based on the fitting results and the regression results, determine whether the target in the image is lost; If the target in the image is not lost, then sub-pixel fitting is performed based on the fitting results and regression results to obtain the true physical deformation of the monitored target; The step of determining the fitting result based on the initial position of the target and determining the regression result based on the multi-scale features includes: Position regression is performed on the multi-scale features to obtain the regression results; The initial position of the target is geometrically fitted, and the scale factor is calculated to obtain the fitting result; The step of performing location regression on the multi-scale features to obtain regression results includes: Non-maximum thresholding is applied to the detection boxes in the target boxes of the multi-scale features whose confidence is greater than a set value to obtain three optimal detection boxes; Calculate the average of the three optimal detection boxes to obtain the regression results; The step of performing sub-pixel fitting based on the fitting results and regression results to obtain the true physical deformation of the monitored target includes: Sub-pixel aggregation is performed based on the fitting and regression results to obtain the final pixel deformation value; The final pixel deformation value is used for scale transformation to obtain the true physical deformation of the monitored target; The step of performing sub-pixel aggregation based on the fitting results and regression results to obtain the final pixel deformation value includes: Calculate the pixel shape variables of the fitting result and the pixel shape variables of the regression result; Calculate the average of the pixel deformation variables of the fitting result and the pixel deformation variables of the regression result to obtain the final pixel deformation value; The process of using the final pixel deformation value for scale transformation to obtain the true physical deformation of the monitored target includes: The true physical deformation of the monitored target is obtained by multiplying the final pixel deformation value by the scale factor.

2. The machine vision-based method for monitoring micro-shake deformation and displacement according to claim 1, characterized in that, The process of performing target detection on the image to obtain the initial target position and multi-scale features includes: The YOLOv7 model is used to perform target detection on the image to obtain the initial target position and multi-scale features.

3. A machine vision-based anti-micro-vibration deformation and displacement monitoring device, characterized in that, include: The image acquisition unit is used to acquire images captured by the camera; The target detection unit is used to perform target detection on the image to obtain the initial position of the target and multi-scale features; The result determination unit is used to determine the fitting result based on the initial position of the target and to determine the regression result based on the multi-scale features. A judgment unit is used to determine whether the target in the image is lost based on the fitting result and the regression result; The subpixel fitting unit is used to perform subpixel fitting based on the fitting result and the regression result if the target in the image is not lost, so as to obtain the true physical deformation of the monitored target. The step of determining the fitting result based on the initial position of the target and determining the regression result based on the multi-scale features includes: Position regression is performed on the multi-scale features to obtain the regression results; The initial position of the target is geometrically fitted, and the scale factor is calculated to obtain the fitting result; The step of performing location regression on the multi-scale features to obtain regression results includes: Non-maximum thresholding is applied to the detection boxes in the target boxes of the multi-scale features whose confidence is greater than a set value to obtain three optimal detection boxes; Calculate the average of the three optimal detection boxes to obtain the regression results; The step of performing sub-pixel fitting based on the fitting results and regression results to obtain the true physical deformation of the monitored target includes: Sub-pixel aggregation is performed based on the fitting and regression results to obtain the final pixel deformation value; The final pixel deformation value is used for scale transformation to obtain the true physical deformation of the monitored target; The step of performing sub-pixel aggregation based on the fitting results and regression results to obtain the final pixel deformation value includes: Calculate the pixel shape variables of the fitting result and the pixel shape variables of the regression result; Calculate the average of the pixel deformation variables of the fitting result and the pixel deformation variables of the regression result to obtain the final pixel deformation value; The process of using the final pixel deformation value for scale transformation to obtain the true physical deformation of the monitored target includes: The true physical deformation of the monitored target is obtained by multiplying the final pixel deformation value by the scale factor.

4. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1 to 2.

5. A storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 2.

Images