Intelligent non-contact structure displacement detection method based on computer vision

Abstract

The invention discloses an intelligent non-contact structure displacement detection method based on computer vision, and belongs to the field of deep learning, and the method comprises the steps: obtaining a first image containing a displacement target, building a first recognition model based on a YOLOv7 algorithm, processing the first image through the first recognition model, and obtaining a recognition result of the displacement target; constructing a tracking network of the displacement target based on a DeepSORT algorithm, and inputting an identification result into the tracking network to form a movement track of the displacement target; a structural displacement analysis network is constructed based on an IPM-ED method, the structural displacement analysis network analyzes the motion trail, and actual structural displacement of the displacement target is generated; and forming a first time-history curve of the displacement target based on the actual structure displacement, and comparing the first time-history curve with the first measurement curve to obtain a structure displacement detection effect. According to the invention, the precision of non-contact displacement detection is improved.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning, and specifically relates to an intelligent non-contact structural displacement detection method based on computer vision. Background Art

[0002] Structural displacement is an important indicator for structural status assessment and performance evaluation. Structural displacement detection methods can be divided into contact detection and non-contact detection. Contact displacement detection usually uses wire displacement sensors, capacitive displacement sensors and acceleration sensors. These sensors have problems such as difficult installation, easy wear and low flexibility, and their promotion and application in actual engineering are limited. Non-contact displacement detection usually uses laser vibrometers, global positioning systems, total stations and microwave radars. These sensors have problems such as high cost, low accuracy and difficulty in achieving multi-point high-frequency continuous measurement.

[0003] Computer vision systems achieve high-precision displacement measurement through visual feature point detection and digital image correlation technology, while deep learning technology further expands the application scenarios of computer vision systems by improving measurement accuracy, analyzing three-dimensional displacement, and replacing the human brain for displacement estimation.

[0004] At present, there is a non-contact structural displacement detection method, for example, a non-contact structural displacement monitoring system is provided in the Chinese patent application document with publication number CN218847128U, including an image acquisition device, which is connected to an image processing system through a data line; and a fluorescent marker plate is arranged on the surface of the structure to be measured, and the fluorescent marker plate is arranged within the image acquisition range of the image acquisition device. The non-contact structural displacement monitoring system can absorb light during the day and release it at night through the additional fluorescent marker plate, so as to provide target tracking information for image processing at night, thereby realizing the displacement monitoring function that takes into account both day and night.

[0005] However, the above-mentioned non-contact structural displacement detection method only realizes the displacement tracking of the target object by setting the fluorescent marker on the target object, and does not generate the dynamic evolution process of the displacement of the target object. In addition, there are problems such as inaccurate structural displacement detection and low flexibility. Therefore, a simple and convenient non-contact displacement detection method that can perform multi-point continuous measurement is needed. Summary of the invention

[0006] To solve the above problems, the present invention provides an intelligent non-contact structural displacement detection method based on computer vision to solve the problems in the prior art.

[0007] In order to achieve the above-mentioned object of the invention, the present invention proposes an intelligent non-contact structural displacement detection method based on computer vision, which comprises:

[0008] Acquire a first image containing a displacement target, establish a first recognition model based on a YOLOv7 algorithm, and use the first recognition model to process the first image to obtain a recognition result of the displacement target;

[0009] Building a tracking network for the displacement target based on the DeepSORT algorithm, inputting the recognition result into the tracking network to form a motion trajectory of the displacement target;

[0010] Building a structural displacement analysis network based on the IPM-ED method, wherein the structural displacement analysis network analyzes the motion trajectory to generate an actual structural displacement of the displacement target;

[0011] A first time history curve of the displacement target is formed based on the actual structural displacement, and the first time history curve is compared with the first measurement curve to obtain a structural displacement detection effect.

[0012] Furthermore, establishing a first recognition model based on the YOLOv7 algorithm includes the following steps:

[0013] The first recognition model includes a Backbone layer, a Neck layer and a Head layer. The first features of different sizes in the first image are extracted based on the BoT-MHSA module in the Backbone layer, and the first features of multiple channels are obtained through the MPCA module. The Neck layer fuses the first features to obtain a first fused feature, and then uses the SimAM attention module to process the first fused feature, and obtains the recognition result of the displacement target through the Head layer.

[0014] Furthermore, constructing the tracking network includes the following steps:

[0015] Based on the recognition result, the DeepSORT algorithm obtains the first bounding box of the displacement target and sets a displacement detection area. Within the displacement detection area, the displacement target ID is assigned. The DeepSORT algorithm performs a first data match based on the first bounding box and the predicted bounding box, and determines whether the data match is successful based on the IOU value. If it fails, the IOU value is reduced to perform a second data match. If the data match is successful, the motion trajectory of the displacement target is updated.

[0016] Furthermore, constructing a structural displacement analysis network based on the IPM-ED method includes the following steps:

[0017] Based on the motion trajectory, the position information of the displacement target is obtained, and a virtual coordinate system is set. The scale information of the virtual coordinate system corresponds to the actual physical size. Based on the IPM method, the position information of the same displacement target is extracted from different image frames, and all the position information is converted into the same virtual coordinate system using a transformation matrix. The actual structural displacement of the displacement target is calculated based on the Euclidean distance.

[0018] Furthermore, setting the displacement detection area includes the following steps:

[0019] The displacement detection area is set as a closed scene. In the closed scene, an ID maximum value is set. If the ID of the displacement target is less than the ID maximum value, the motion trajectory of the displacement target is updated, otherwise, no new motion trajectory is generated.

[0020] Further, judging whether the data matching is successful based on the IOU value includes the following steps:

[0021] The IOU value is the overlap degree between the first bounding box and the predicted bounding box. A second threshold is set. If the IOU value is greater than the second threshold, the first bounding box and the predicted bounding box data are matched successfully, otherwise the match fails.

[0022] Furthermore, the predicted bounding box is predicted and generated based on the displacement change of the displacement target.

[0023] Furthermore, the ID maximum value is an average value of the number of displacement targets detected in nearly three frames of images in the displacement detection area.

[0024] Furthermore, the recognition result includes coordinate information and category information.

[0025] Furthermore, the first measurement curve is acquired based on a contact sensor device.

[0026] Compared with the prior art, the present invention has at least the following beneficial effects:

[0027] In view of the problems of high cost, low precision and poor flexibility of existing displacement detection methods, the present invention proposes a real-time detection method for structural displacement based on computer vision and deep learning. Firstly, high-precision displacement target recognition and detection is achieved through the improved YOLOv7 algorithm; then, based on the trajectory generation and data matching method in the improved DeepSORT algorithm, the stability of the displacement target tracking process is improved; at the same time, a virtual coordinate system is established based on the displacement target, and the structural displacement is analyzed and converted based on the Euclidean distance of the inverse perspective transformation. The model constructed by the present invention can significantly improve the displacement detection accuracy, accurately capture the dynamic evolution process of the structural displacement, and realize the intelligent real-time recognition of the structural displacement. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A flowchart of the steps of a computer vision-based intelligent non-contact structural displacement detection method of the present invention;

[0029] Figure 2 It is a model structure diagram of the present invention;

[0030] Figure 3 A diagram showing the displacement target setting of the present invention;

[0031] Figure 4 It is a visual display diagram of the displacement target detection test of the present invention;

[0032] Figure 5 is a first time course graph of the present invention;

[0033] Figure 6 The cable dome structure and test arrangement diagram of the present invention;

[0034] Figure 7 It is a visual display diagram of the cable breaking test of the cable dome structure of the present invention;

[0035] Figure 8 is a second time course graph of the present invention;

[0036] Fig. 9 It is a network structure diagram of the improved YOLOv7 algorithm of the present invention;

[0037] Fig.10 This is a network structure diagram of the improved DeepSORT algorithm of the present invention;

[0038] Fig.11 It is the structural displacement analysis diagram of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0040] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0041] like Figure 1As shown, an intelligent non-contact structural displacement detection method based on computer vision, the method comprising:

[0042] S1: Acquire a first image containing a displacement target, establish a first recognition model based on the YOLOv7 algorithm, use the first recognition model to process the first image, and obtain a recognition result of the displacement target.

[0043] Specifically, in this embodiment, the displacement detection of the displacement target in the first image is mainly divided into three stages: recognition, tracking and displacement analysis. Figure 2 As shown, it is a model structure diagram of the present invention, which recognizes and detects the displacement target by improving the YOLOv7 algorithm to obtain the recognition result, which includes the coordinate information and category information of the displacement target.

[0044] The first image containing the displacement target is collected at the test site to form a data set. The collected data types include targets attached to the surface of the test piece and targets placed in the test site. The data collection device is a smart phone. The data set of this embodiment includes 129,360 frames of video images. In order to improve the robustness of the training results, some pictures are randomly extracted from each video to obtain the first image. In order to facilitate the distinction and display of the detection effects of different displacement measurement points, such as Figure 3 As shown in the figure, it is a displacement target setting diagram. Four different types of displacement targets are designed and marked as label_1, label_2, label_3 and label_4 respectively.

[0045] S2: Construct a tracking network for the displacement target based on the DeepSORT algorithm, input the recognition results into the tracking network, and form the motion trajectory of the displacement target.

[0046] Specifically, in this embodiment, an improved DeepSORT algorithm is used to perform matching tracking of the displacement target, data association is performed across frames to form the motion trajectory of the displacement target, and the tracked trajectory information is transmitted to the structural displacement analysis network.

[0047] S3: Based on the IPM-ED method, a structural displacement analysis network is constructed. The structural displacement analysis network analyzes the motion trajectory and generates the actual structural displacement of the displacement target.

[0048] Specifically, in this embodiment, the structural displacement parsing network integrates a large amount of spatiotemporal information in the continuous image sequence and performs actual structural displacement of the displacement target based on the IPM-ED method.

[0049] S4: forming a first time history curve of the displacement target based on the actual structural displacement, and comparing the first time history curve with the first measurement curve to obtain a structural displacement detection effect.

[0050] Specifically, in this embodiment, Figure 4 As shown in the figure, it is a visual display diagram of the displacement target detection test. Taking 4 bamboos as an example, 4 displacement targets are set on each bamboo. The overall displacement of the measuring point is displayed in real time above each displacement target in the detection screen, and it is green when the overall displacement is less than 50mm, yellow when the overall displacement is between 50-100mm, and red when the overall displacement is greater than 100mm. In addition, the real-time displacement and overall displacement of each displacement target are displayed in real time in the upper left corner of each detection screen. During the test, the maximum displacement appeared in the middle of the bamboo and gradually decreased towards the top and bottom. The change trend of the lateral displacement showed a state of first increasing and then weakening. At the beginning of the test, the displacement of the bamboo increased rapidly. When the yield load was reached, the displacement increased slowly.

[0051] like Figure 5 As shown, it is the first time course curve diagram of the four displacement target measuring points (label_1, label_2, label_3 and label_4) in the bamboo 1, wherein the red curve is the displacement change curve detected by the method of the present invention, and the blue curve is the first measurement curve, i.e., the data measured by the displacement sensor. The displacement recognition accuracy of label_1 to label_4 is high, and the first time course curve is highly consistent with the sensor measurement data. After removing the measurement error of the first time course data in the experiment and the disturbance error of the lens during shooting, it can be considered that the result of displacement detection has achieved a high accuracy.

[0052] As can be seen from Table 1, the error percentages between the detected peak displacement and the actual peak displacement are within 5%, especially the peak displacement error percentage of label_4 is only 0.38%, indicating that the model used in the present invention has high accuracy. In addition, the minimum value of the peak displacement of label_1 to label_4 is 29.36mm, and the maximum value is 158.17mm, which is nearly 6 times different. The average error is maintained at about 2.50mm, and the average errors are 2.52mm, 3.10mm, 2.67mm, and 1.60mm, respectively, indicating that the peak value of the displacement of the measuring point has a low impact on the detection accuracy, the detection model has high stability, and the actual displacement and the detected displacement have a high degree of fit.

[0053] Table 1 Displacement detection results of bamboo 1

[0054]

[0055] As another embodiment of the present invention, Figure 6The cable dome structure and test arrangement are shown in FIG. The cable dome structure is made of steel. The cable is broken by an angle grinder. The displacement of key nodes before and after the cable is broken is detected by the method of the present invention. There are 4 test points in total, label_1 to label_4.

[0056] like Figure 7 The figure below shows a visual display of the cable dome structure cable breaking test. The overall displacement of the measuring point is displayed in real time above each displacement target in the test screen.

[0057] like Figure 8 As shown in the figure, it is the second time history curve of the four measuring points in the cable dome structure cable breaking test. When the angle grinder breaks the cable, the node displacement will change suddenly, and the sudden break of the cable will cause the structure to vibrate up and down. After a period of time, the structure tends to be stable, and the node displacement also gradually tends to be stable. At other times, the node displacement remains stable.

[0058] The displacement detection model based on the computer vision method of the present invention has high accuracy and good feasibility in non-contact detection of displacement, realizes non-contact, efficient and intelligent recognition of structural displacement, and has great application potential.

[0059] As a preferred technical solution of the present invention, establishing a first recognition model based on the YOLOv7 algorithm includes the following steps:

[0060] The first recognition model includes Backbone layer, Neck layer and Head layer. The BoT-MHSA module in the Backbone layer extracts the first features of different sizes in the first image, and obtains the first features of multiple channels through the MPCA module. The Neck layer fuses the first features to obtain the first fused features, and then uses the SimAM attention module to process the first fused features, and obtains the recognition results of the displacement targets through the Head layer.

[0061] Specifically, in this embodiment, Fig. 9As shown in the figure, it is the improved YOLOv7 algorithm network structure diagram. The first recognition model constructed based on the YOLOv7 algorithm includes Backbone layer, Neck layer and Head layer. Since only the Backbone layer and Neck layer are improved, the Head layer is not displayed. The dotted box part is the improved part of the YOLOv7 algorithm. First, the BoT-MHSA module is added to the Backbone and Neck to improve the recognition ability of the first recognition model for small-sized targets, and the ELAN module behind the BoT-MHSA module is deleted to improve the operation speed of the first recognition model. Then, the MP module in the YOLOv7 algorithm is replaced by the MPCA module to capture cross-channel and cross-space feature information, and retain the perceptual information of the position coordinates. Finally, the SimAM attention mechanism is used before feature decoding to correct the deviation of the network's area of ​​interest, and the focus area of ​​the target is obtained from a large amount of global information, further reducing the missed detection rate and false detection rate of the first recognition model.

[0062] Building a tracking network consists of the following steps:

[0063] Based on the recognition results, the DeepSORT algorithm obtains the first bounding box of the displacement target and sets the displacement detection area. Within the displacement detection area, the displacement target ID is assigned. The DeepSORT algorithm performs a first data match based on the first bounding box and the predicted bounding box, and determines whether the data match is successful based on the IOU value. If it fails, the IOU value is reduced for the second data match. If the data match is successful, the motion trajectory of the displacement target is updated.

[0064] Specifically, in this embodiment, Fig.10 As shown in the figure, it is a network structure diagram of the improved DeepSORT algorithm. DeepSORT is a detection-based multi-target tracking algorithm. Based on the position information of the displacement target in the recognition result, the first bounding box of the displacement target is obtained, and the predicted bounding box is predicted and generated based on the displacement change of the displacement target. In the process of matching the first bounding box and the predicted bounding box, the original DeepSORT algorithm only performs one data matching (first data matching) and only completes the preliminary matching of the displacement target. In the multi-target tracking task, it is common for the target to be blocked or temporarily lost. If only one matching is performed, the algorithm may not be able to effectively handle these situations, resulting in target tracking loss or generating erroneous tracking trajectories. The improved DeepSORT algorithm adds an IOU matching (second data matching) to process the detection box that did not match the first data matching successfully, reduce the IOU value, relax the limit of the maximum distance of the intersection-union ratio, and make the unmatched detection box successfully match the trajectory as much as possible, so as to obtain the motion trajectory of the displacement target.

[0065] The construction of the structural displacement analysis network based on the IPM-ED method includes the following steps:

[0066] The position information of the displacement target is obtained based on the motion trajectory, and a virtual coordinate system is set. The scale information of the virtual coordinate system corresponds to the actual physical size. The position information of the same displacement target is extracted from different image frames based on the IPM method. All the position information is converted to the same virtual coordinate system using the transformation matrix, and the actual structural displacement of the displacement target is calculated based on the Euclidean distance.

[0067] Specifically, in this embodiment, during the displacement analysis process, the camera will inevitably have a certain angle and the lens is difficult to face all displacement targets. The existence of the angle will cause objects of the same size to appear as different sizes in the video image, which will cause serious displacement detection errors. In addition, the pixel displacement in the video image needs to be converted into real displacement. The calibration of the conversion relationship in different scenes is relatively cumbersome, and sometimes it is even difficult to find a suitable size calibration standard.

[0068] The present invention constructs a set of virtual coordinate systems based on the IPM-ED method, such as Fig.11 As shown in the figure, the structural displacement analysis diagram is used to convert pixel coordinates to virtual coordinates using inverse perspective transformation (IPM). The scale of the virtual coordinate system is determined by a target of known physical size, so that the scale information on the virtual coordinate system is consistent with the actual physical size. Finally, the coordinate information of the same target is extracted in different frames, and the displacement of the Euclidean distance (ED) is calculated after mapping by the transformation matrix, thereby reducing the detection error of the structural displacement under perspective photography and obtaining a more accurate actual structural displacement.

[0069] Setting the displacement detection area includes the following steps:

[0070] Set the displacement detection area to a closed scene. In the closed scene, set the ID maximum value. If the ID of the displacement target is less than the ID maximum value, update the motion trajectory of the displacement target. Otherwise, do not generate a new motion trajectory.

[0071] Specifically, in this embodiment, the displacement detection area is a closed specific scene, the total number of displacement targets in the closed scene is fixed, and the maximum value of ID is known. Therefore, the growth of ID can be limited by improving the generation method of the trajectory in the closed scene. In actual tracking, the maximum value of ID is set to the total number of targets in the displacement target detection result. Since there is a certain probability of false detection and missed detection in the detection result, the maximum value of ID is determined based on the average value of the number of displacement targets detected in the last three frames of the first image where the current displacement target is located. If the ID value of the currently generated trajectory exceeds the maximum value of ID, no new trajectory is generated. If it does not exceed, a new trajectory is generated.

[0072] Determining whether data matching is successful based on the IOU value includes the following steps:

[0073] The IOU value is the degree of overlap between the first bounding box and the predicted bounding box. A second threshold is set. If the IOU value is greater than the second threshold, the first bounding box and the predicted bounding box data are matched successfully, otherwise the match fails.

[0074] Compared with the prior art, the present invention has at least the following beneficial effects:

[0075] In view of the problems of high cost, low precision and poor flexibility of existing displacement detection methods, the present invention proposes a real-time detection method for structural displacement based on computer vision and deep learning. Firstly, high-precision displacement target recognition and detection is achieved through the improved YOLOv7 algorithm; then, based on the trajectory generation and data matching method in the improved DeepSORT algorithm, the stability of the displacement target tracking process is improved; at the same time, a virtual coordinate system is established based on the displacement target, and the structural displacement is analyzed and converted based on the Euclidean distance of the inverse perspective transformation. The model constructed by the present invention can significantly improve the displacement detection accuracy, accurately capture the dynamic evolution process of the structural displacement, and realize the intelligent real-time recognition of the structural displacement.

[0076] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0077] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The above-mentioned program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0078] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0080] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent non-contact structural displacement detection method based on computer vision, characterized in that: The method comprises the following steps: Acquire a first image containing a displacement target, establish a first recognition model based on a YOLOv7 algorithm, and use the first recognition model to process the first image to obtain a recognition result of the displacement target; Building a tracking network for the displacement target based on the DeepSORT algorithm, inputting the recognition result into the tracking network to form a motion trajectory of the displacement target; Building a structural displacement analysis network based on the IPM-ED method, wherein the structural displacement analysis network analyzes the motion trajectory to generate an actual structural displacement of the displacement target; A first time history curve of the displacement target is formed based on the actual structural displacement, and the first time history curve is compared with the first measurement curve to obtain a structural displacement detection effect.

2. The method according to claim 1, characterized in that Establishing the first recognition model based on the YOLOv7 algorithm includes the following steps: The first recognition model includes a Backbone layer, a Neck layer and a Head layer. The first features of different sizes in the first image are extracted based on the BoT-MHSA module in the Backbone layer, and the first features of multiple channels are obtained through the MPCA module. The Neck layer fuses the first features to obtain a first fused feature, and then uses the SimAM attention module to process the first fused feature, and obtains the recognition result of the displacement target through the Head layer.

3. The method according to claim 2, characterized in that Building the tracking network includes the following steps: Based on the recognition result, the DeepSORT algorithm obtains the first bounding box of the displacement target and sets a displacement detection area. Within the displacement detection area, the displacement target ID is assigned. The DeepSORT algorithm performs a first data match based on the first bounding box and the predicted bounding box, and determines whether the data match is successful based on the IOU value. If it fails, the IOU value is reduced to perform a second data match. If the data match is successful, the motion trajectory of the displacement target is updated.

4. The method according to claim 3, characterized in that The construction of the structural displacement analysis network based on the IPM-ED method includes the following steps: Based on the motion trajectory, the position information of the displacement target is obtained, and a virtual coordinate system is set. The scale information of the virtual coordinate system corresponds to the actual physical size. Based on the IPM method, the position information of the same displacement target is extracted from different image frames, and all the position information is converted into the same virtual coordinate system using a transformation matrix. The actual structural displacement of the displacement target is calculated based on the Euclidean distance.

5. The method according to claim 3, characterized in that: Setting the displacement detection area includes the following steps: The displacement detection area is set as a closed scene. In the closed scene, an ID maximum value is set. If the ID of the displacement target is less than the ID maximum value, the motion trajectory of the displacement target is updated, otherwise, no new motion trajectory is generated.

6. The method according to claim 3, characterized in that Determining whether data matching is successful based on the IOU value includes the following steps: The IOU value is the degree of overlap between the first bounding box and the predicted bounding box. A second threshold is set. If the IOU value is greater than the second threshold, the first bounding box and the predicted bounding box data are matched successfully, otherwise the match fails.

7. The method according to claim 3, characterized in that The predicted bounding box is predicted and generated based on the displacement change of the displacement target.

8. The method according to claim 5, characterized in that The maximum ID value is the average value of the number of displacement targets detected in the displacement detection area in the last three frames of images.

9. The method according to claim 1, characterized in that: The recognition result includes coordinate information and category information.

10. The method according to claim 1, characterized in that The first measurement curve is acquired based on a contact sensor device.

Images