Outdoor equipment detection data acquisition method, system and device and storage medium

Abstract

The invention discloses an outdoor equipment detection data acquisition method, system and device and a storage medium. The method comprises the following steps: acquiring video data in a visual area; identifying a detection area of each frame of image; pre-storing each frame of image in a cache region; comparing the identification condition of the detection area of the intermediate image with the identification conditions of the detection areas in other images in the current cache area, judging whether the intermediate image has a detection error or not, and storing the intermediate image with the detection error and the position information of the detection area of the intermediate image into a sample set; and data stored in the sample set is transmitted to a data center, and then the sample set is emptied. The method of consuming a large amount of data streams such as real-time data transmission is avoided, and key data can be stored in a scene without signals; and meanwhile, the problem of misjudgment of equipment control caused by too long breakpoint identification time is avoided, and the safety performance of equipment operation is ensured.

Description

Technical Field

[0001] The present invention relates to the fields of agricultural machinery data detection and engineering machinery data detection, and in particular to a method, system, device and storage medium for collecting detection data of outdoor equipment. Background Art

[0002] The monitoring of equipment operation usually requires the transmission of a large amount of data. In the prior art, the operation of the equipment is monitored in real time through video, such as a remote control device for agricultural machinery with video monitoring disclosed in the utility model patent with authorization announcement number CN202013513U. The remote control device with video monitoring can achieve off-site real-time remote control through real-time video monitoring; it is used for remote control of agricultural machinery and real-time monitoring of the operation effect. At the same time, intelligent engineering machinery equipment and agricultural equipment also need to collect and identify relevant image data around the equipment during operation, and refer to the recognition results for automated operation. However, during the operation of the equipment, the equipment and the object to be identified are in a relative motion state, so the position and shape of the object collected by the visual equipment are in a real-time changing process. In this changing process, when the sample data of the model training is insufficient, the object cannot be identified at some time, which leads to poor recognition continuity of the target detection object and the phenomenon of recognition breakpoints; when the recognition breakpoint time is too long, it may cause equipment control to make judgment errors or even cause accidents, so it is necessary to gradually optimize the data samples to solve the problem of identifying sampling breakpoints.

[0003] In addition, in the field of construction machinery or agricultural machinery, since the equipment has a variety of working conditions, it brings a large amount of data to be processed. It is difficult for existing data set models to accurately identify and process these work scenarios with large amounts of data; at the same time, since the working scenes of construction machinery and agricultural machinery are usually outdoors, it is difficult to collect network data sets. In order to reduce costs, various engineering and agricultural machinery equipment are only equipped with some low-speed 4G networks with limited traffic, which makes it difficult to collect data in real time and transmit it to the server; in many cases, the equipment even works in places without signals, further increasing the difficulty of data collection. Summary of the invention

[0004] Therefore, in order to solve the above problems, the present invention provides a method, system, device and storage medium for collecting outdoor equipment detection data.

[0005] The present invention is achieved through the following technical solutions: The outdoor equipment detection data collection method comprises the following steps: Collecting video data in a visible area, wherein the video data includes multiple frames of images; Identify the detection area of ​​each frame of the image, where the detection area is the area where the target detection object is displayed in the current image; Starting from the first frame image of the video data, each frame image is pre-stored in the buffer area in sequence, so that the number of images stored in the buffer area reaches a preset number; the preset number is an odd number, and a frame image located in the middle position of the buffer area is the middle image of the current buffer area; Compare the recognition status of the detection area of ​​the intermediate image with the recognition status of the detection areas of other images in the current buffer area, determine whether the intermediate image has a detection error, and store the intermediate image with the detection error and the position information of the detection area of ​​the intermediate image in the sample set; The sample set is checked in real time based on the preset frequency to see if there is data in it, and the sample set is cleared after the data stored in the sample set is transmitted to the data center.

[0006] Preferably, the “comparing the recognition status of the detection area of ​​the intermediate image with the recognition status of the detection areas of other images in the current buffer area to determine whether a detection error occurs in the intermediate image” includes: If the detection area is not recognized in the intermediate image in the current buffer, and the detection area is recognized in other images before or after the intermediate image in the current buffer, it is determined that a first detection error occurs in the current intermediate image; If the intermediate image in the current buffer area recognizes the detection area, and other images in the current buffer area do not recognize the detection area, it is determined that a second detection error occurs in the intermediate image.

[0007] Preferably, if the first detection error occurs in the intermediate image of the current cache area, the image of the most recent frame before and after the intermediate image in the current cache area that can identify the detection area is found respectively, and the position information of the detection area in the two frames is obtained respectively. The position information of the detection area in the two frames is smoothed to calculate the position information of the detection area in the current intermediate image, and then the current intermediate image and the calculated position information of the detection area are stored in the positive sample set.

[0008] Preferably, if a second detection error occurs in the intermediate image of the current buffer area, the position information of the detection area in the current intermediate image is obtained, and the position information of the current intermediate image and its detection area is stored in the negative sample set.

[0009] Preferably, the "detecting in real time whether there is data in the sample set according to a preset frequency, and clearing the sample set after transmitting the data stored in the sample set to the data center" means detecting in real time whether there is stored data in the positive sample set and the negative sample set according to a preset frequency, and if there is stored data in the sample set, transmitting the data to the data center and clearing the transmitted stored data.

[0010] Preferably, if there is no detection error in the current intermediate image, the image in the buffer area is directly cleared; If there is a detection error in the intermediate image, the intermediate image is stored in the sample set and the image in the buffer is cleared; After the image in the buffer is cleared, the image is continuously stored in the buffer starting from the next frame of the cleared image.

[0011] Preferably, the location information includes the center point coordinates of the detection area and its width and height data.

[0012] Outdoor equipment detection data acquisition system, including: Visual equipment, used to collect video data of the target detection object within the visible area; An identification module, used to identify a detection area of ​​each frame of the video data and obtain location information of the detection area, wherein the detection area is an area where the target detection object is displayed in the current image; The buffer area is used to periodically pre-store each frame of the video data, and output the intermediate image and its position information to the sample collection module and clear the data after processing the current pre-stored image each time, and then continue to pre-store the next batch of images; the preset number is an odd number, and the frame of image located in the middle position of the buffer area is the intermediate image of the current buffer area; A logic control module determines whether a first detection error and a second detection error occur in an intermediate image in a buffer area, wherein the first detection error refers to that the intermediate image in the current buffer area fails to identify a detection area, and the detection area is identified in other images before or after the intermediate image in the buffer area; the second detection error refers to that the intermediate image in the current buffer area identifies a detection area, and the detection area is not identified in other images in the buffer area; if a first detection error occurs in an intermediate image in the current buffer area, the image of the most recent frame that identifies the detection area is found before and after the intermediate image, respectively, and the position information of the detection area in the two frames of images is obtained respectively, and the position information of the detection area of ​​the current intermediate image is calculated by smoothing the two position information; The sample collection module includes a positive sample set and a negative sample set. The positive sample set is used to collect intermediate images with a first detection error in a cache area and the detection area position information obtained after calculation. The negative sample set is used to collect intermediate images with a second detection error in the cache area and the position information thereof. The positive sample set and the negative sample set each transmit their data to a data center and then clear the data.

[0013] The outdoor equipment detection data collection device comprises the outdoor equipment detection data collection system as described above.

[0014] The storage medium includes the outdoor equipment detection data acquisition system as described above.

[0015] The beneficial effects of the technical solution of the present invention are mainly reflected in: 1. In this method, only the detection error data is stored and transmitted, and a large amount of invalid data is discarded and cleared to achieve extremely small data storage capacity, avoiding the practice of consuming a large amount of data flow such as real-time data transmission. At the same time, even if the equipment works in a scene without signal, it can store key data in a small capacity locally, which is convenient for collecting large quantities of data distributed in various construction sites, fields and other outdoor scenes to achieve model optimization.

[0016] 2. Pre-store images in the cache area in sequence, find the middle image in the cache area and compare it with the images before and after it, so as to quickly compare and determine whether a detection error occurs. The middle image caused by the breakpoint due to the first detection error is restored through smoothing, thereby avoiding the problem of misjudgment of equipment control caused by too long breakpoint recognition time, and ensuring the safety performance of equipment operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the outdoor equipment detection data collection method; Figure 2 It is a workflow diagram for collecting positive sample data in the outdoor equipment detection data collection method; Figure 3 It is a workflow diagram for collecting negative sample data in the outdoor equipment detection data collection method; Figure 4 It is a workflow diagram for transferring data from the positive sample set and the negative sample set to the data center. DETAILED DESCRIPTION

[0018] In order to make the purpose, advantages and features of the present invention more clearly and in detail, the following preferred embodiments are used for illustration and explanation. The embodiments are only typical examples of the application of the technical solution of the present invention. Any technical solution formed by equivalent replacement or equivalent transformation falls within the scope of protection claimed by the present invention.

[0019] At the same time, it is stated that in the description of the scheme, it should be noted that the orientation or position relationship indicated by the terms "middle", "front", "rear", etc. is based on the orientation or position relationship shown in the drawings, and is only for the convenience of description and simplified description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0020] In addition, the terms "first" and "second" in this solution are only used for descriptive purposes and cannot be understood as indicating or implying the order of importance, or implicitly indicating the number of technical features shown. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0021] The present invention discloses a method for collecting outdoor equipment detection data, such as Figure 1 As shown, the following steps are included: Collect video data within the visible area, the video data includes multiple frames of images; the video data can be captured by a visual device, and the visual device can use an existing visual device with large-scene camera. In addition, the same outdoor device can use multiple visual devices to monitor the target detection object at different angles and different areas, which will not be elaborated here.

[0022] Identify the detection area of ​​each frame image, the detection area is the area where the target detection object is displayed in the current image; in one embodiment, if the detection area in the current image is not identified, mark false in the image; Starting from the first frame image of the video data, each frame image is pre-stored in the buffer area in sequence, so that the number of images stored in the buffer area reaches a preset number; wherein the preset number is an odd number (and the preset number in the buffer area should be 2n+1, n≥1), and a frame image located in the middle position of the buffer area is the middle image of the current buffer area; in one embodiment, the preset number is 5, so the buffer area collects 5 consecutive frames of images each time, at this time, when the buffer area is full of 5 frames of images, the middle image is the third frame image, and there are two frames of images before and after the middle image; Compare the recognition status of the detection area of ​​the intermediate image with the recognition status of the detection areas of other images in the current buffer area, determine whether the intermediate image has a detection error, and store the intermediate image with the detection error and the position information of the detection area of ​​the intermediate image in the sample set; The sample set is checked in real time based on the preset frequency to see if there is data in it, and the sample set is cleared after the data stored in the sample set is transmitted to the data center.

[0023] Wherein, the recognition status of the detection area in each frame image includes a recognizable detection area and an unrecognizable detection area. If the current image can recognize the detection area, the position data of the detection area in the current image is further obtained; in some embodiments, the "checking the recognition status of the detection area of ​​the intermediate image and determining whether a detection error occurs in the detection area of ​​the intermediate image" includes the following steps: If the detection area is not recognized in the intermediate image in the current buffer (i.e., the false mark in the intermediate image is detected), and the detection area is recognized in other images before or after the intermediate image in the current buffer, it is determined that the first detection error occurs in the current intermediate image; The judgment process is as follows: Referring to the above embodiment, when the preset number of images in the buffer area is 5, if the detection area of ​​the intermediate image (the third frame image) is not identified, the following situations exist: First, the detection area is not recognized in the first two frames of the intermediate image, and it is determined that the target object has left the visible area of ​​the current visual device; Secondly, when at least one of the first two frames of the intermediate image can identify the detection area, and at least one of the second two frames of the intermediate image can also identify the detection area, it is determined that the first detection error occurs in the current intermediate image; Third, if at least one of the first two frames of the intermediate image can identify the detection area, and the detection area is not identified in the last two frames of the intermediate image, it is judged that the target detection object has re-entered the visible area of ​​the current visual device.

[0024] like Figure 2As shown, in one embodiment, if the first detection error occurs in the intermediate image of the current buffer area, the image of the most recent frame before and after the intermediate image in the current buffer area that can identify the detection area is found respectively, and the position information of the detection area in the two frames of images is obtained respectively, and the position information of the detection area in the two frames of images is smoothed to calculate the position information of the detection area in the current intermediate image, and then the current intermediate image and the calculated position information of the detection area are stored in the positive sample set; specifically, with reference to the above embodiment, if the first detection error occurs in the intermediate image (the third frame of image), the image cannot be detected at this time. , and at least one of the first and second frames of images in the buffer can identify the detection area, and at least one of the fourth and fifth frames of images can identify the detection area; assuming that the first, fourth, and fifth frames of images can all identify the detection area, but the second frame of image cannot identify the detection area, at this time, the image that can identify the detection area in the most recent frame before the intermediate image is the first frame of image, and the image that can identify the detection area in the most recent frame after the intermediate image is the fourth frame of image, and the position information of the detection area in the first and fourth frames of image is smoothed and calculated to obtain the position information of the detection area in the intermediate image. In one embodiment, the position information includes the center point coordinates (x, y) of the detection area and its width (w) and height (h) data, and at this time, the position information P of the detection area is (x, y, w, h). In one embodiment, according to the characteristics of linear transition of the object position, the smoothing processing adopts the mean processing method. Referring to the above embodiment, when the position information of the detection area in the first frame image is P1 (x1, y1, w1, h1), and the position information of the detection area in the fourth frame image is P4 (x4, y4, w4, h4), then through the mean calculation, the position information of the detection area in the intermediate image is P3 ((x1+x4) / 2, (y1+ y 4) / 2, (w1+ w 4) / 2, (h1+ h4) / 2).

[0025] If the intermediate image in the current cache area recognizes the detection area, and other images in the current cache area do not recognize the detection area, it is judged that the intermediate image has a second detection error; wherein, a negative sample cache area for caching negative samples can be separately set up, and the negative sample cache area is consistent with the caching method of the above-mentioned cache area. Starting from the first frame image of the video data, each frame image is pre-stored in the negative sample cache area in sequence until the number of images stored in the negative sample cache area reaches a preset number, and the recognition status of the detection area of ​​the intermediate image in the current negative sample cache area is detected. If the detection area can be recognized (can be marked as true), and other images in the current negative sample cache area do not recognize the detection area, it is judged that the intermediate image has a second detection error, that is, a negative sample detection error.

[0026] like Figure 3 As shown, in one embodiment, if a second detection error occurs in the intermediate image of the current cache area, the position information of the detection area in the current intermediate image is obtained, and the position information of the current intermediate image and its detection area is stored in the negative sample set. Combined with the information storage of the positive sample set, the accuracy of subsequent data processing can be improved.

[0027] In one embodiment, if the current intermediate image does not have a detection error, the image in the buffer area is directly cleared; referring to the above embodiment, when the intermediate image does not have the first detection error and the second detection error, it indicates that it is in a normal monitoring state, and the image in the buffer area can be directly cleared; If there is a detection error in the intermediate image, the intermediate image is stored in the sample set and the image in the buffer area is cleared; wherein, if the intermediate image has a first detection error, the intermediate image and the position information of the detection area calculated therefrom are synchronously stored in the positive sample set, and the buffer area is cleared for collecting the next batch of images; if the intermediate image has a second detection error, the intermediate image and the position information of the detection area are synchronously stored in the negative sample set, and the negative sample buffer area is cleared; After the cache image is cleared, the image is continuously stored in the cache starting from the next frame of the cleared image; referring to the above embodiment, when the preset number is 5, the cache stores the first to fifth frames of the video data during the initial operation; after the current five frames of the image are processed, the cache is cleared, and the sixth to tenth frames of the video data are stored in sequence; and so on, until the device completes the work.

[0028] like Figure 4 As shown, in one embodiment, the "detecting in real time whether there is data in the sample set according to a preset frequency, and clearing the sample set after transmitting the data stored in the sample set to a data center" means detecting in real time whether there is stored data in the positive sample set and the negative sample set according to a preset frequency. If at least one of the positive sample set and the negative sample set has stored data, the sample set transmits the data to the data center and clears the transmitted stored data.

[0029] The present invention also discloses an outdoor equipment detection data acquisition system, comprising: Visual equipment, used to collect video data of the target detection object within the visible area; An identification module, used to identify a detection area of ​​each frame of the video data and obtain location information of the detection area, wherein the detection area is an area where the target detection object is displayed in the current image; The buffer area is used to periodically pre-store each frame of the video data, and output the intermediate image and its position information to the sample collection module and clear the data after processing the current pre-stored image each time, and then continue to pre-store the next batch of images; the preset number is an odd number, and a frame of image located in the middle of the buffer area is the intermediate image of the current buffer area; the buffer area can adopt a circuit with storage function, such as RAM, FIFO, etc.; it can also adopt a memory in a physical form, such as a memory stick, TF card, etc. This is a prior art and will not be described in detail; A logic control module determines whether a first detection error and a second detection error occur in an intermediate image in a buffer area, wherein the first detection error refers to that the intermediate image in the current buffer area fails to identify a detection area, and the detection area is identified in other images before or after the intermediate image in the buffer area; the second detection error refers to that the intermediate image in the current buffer area identifies a detection area, and the detection area is not identified in other images in the buffer area; if a first detection error occurs in an intermediate image in the current buffer area, the image of the most recent frame that identifies the detection area is found before and after the intermediate image, respectively, and the position information of the detection area in the two frames of images is obtained respectively, and the position information of the detection area of ​​the current intermediate image is calculated by smoothing the two position information; The sample collection module includes a positive sample set and a negative sample set. The positive sample set is used to collect intermediate images with a first detection error in a cache area and the detection area position information obtained after calculation. The negative sample set is used to collect intermediate images with a second detection error in the cache area and the position information thereof. The positive sample set and the negative sample set each transmit their data to a data center and then clear the data.

[0030] In one embodiment, the outdoor equipment detection data collection system also includes a 4G transmission module for detecting whether there is a signal and whether the signal is stable; after the server is successfully connected, the text in the positive sample set and the negative sample set is compressed and transmitted to the data center.

[0031] The present invention also discloses an outdoor equipment detection data acquisition device, including the outdoor equipment detection data acquisition system as described above.

[0032] The present invention also discloses a storage medium, including the outdoor equipment detection data acquisition system as described above.

[0033] There are many implementation methods of the present invention, and all technical solutions formed by equivalent transformation or equivalent transformation fall within the protection scope of the present invention.

Claims

1. The outdoor equipment detection data collection method is characterized by: The following steps are involved: Collecting video data in a visible area, wherein the video data includes multiple frames of images; Identify the detection area of ​​each frame of the image, where the detection area is the area where the target detection object is displayed in the current image; Starting from the first frame image of the video data, pre-storing each frame image in the buffer area in sequence, so that the number of image storages in the buffer area reaches a preset number; The preset number is an odd number, and a frame of image located in the middle of the buffer area is the middle image of the current buffer area; Compare the recognition status of the detection area of ​​the intermediate image with the recognition status of the detection areas of other images in the current buffer area, determine whether the intermediate image has a detection error, and store the intermediate image with the detection error and the position information of the detection area of ​​the intermediate image in the sample set; The sample set is checked in real time based on the preset frequency to see if there is data in it, and the sample set is cleared after the data stored in the sample set is transmitted to the data center.

2. The outdoor equipment detection data collection method according to claim 1, characterized in that: The “checking the recognition status of the detection area of ​​the intermediate image and determining whether a detection error occurs in the detection area of ​​the intermediate image” includes: If the detection area is not recognized in the intermediate image in the current buffer, and the detection area is recognized in other images before or after the intermediate image in the current buffer, it is determined that a first detection error occurs in the current intermediate image; If the intermediate image in the current buffer area recognizes the detection area, and other images in the current buffer area do not recognize the detection area, it is determined that a second detection error occurs in the intermediate image.

3. The outdoor equipment detection data collection method according to claim 2, characterized in that: If the first detection error occurs in the intermediate image of the current cache area, find the image of the most recent frame before and after the intermediate image in the current cache area that can identify the detection area, and obtain the position information of the detection area in the two frames of images respectively. After smoothing the position information of the detection area in the two frames of images, calculate the position information of the detection area in the current intermediate image, and then store the current intermediate image and the calculated position information of the detection area in the positive sample set.

4. The outdoor equipment detection data collection method according to claim 3 is characterized in that: If the second detection error occurs in the intermediate image in the current buffer area, the position information of the detection area in the current intermediate image is obtained, and the position information of the current intermediate image and its detection area is stored in the negative sample set.

5. The outdoor equipment detection data collection method according to claim 4 is characterized in that: The "real-time detection of whether there is data in the sample set according to a preset frequency, and clearing the sample set after transmitting the data stored in the sample set to the data center" means real-time detection of whether there is stored data in the positive sample set and the negative sample set according to a preset frequency. If there is stored data in the sample set, the data is transmitted to the data center and the transmitted stored data is cleared.

6. The outdoor equipment detection data collection method according to claim 1, characterized in that: If there is no detection error in the current intermediate image, the image in the buffer area is directly cleared; If there is a detection error in the intermediate image, the intermediate image is stored in the sample set and the image in the buffer is cleared; After the image in the buffer is cleared, the image is continuously stored in the buffer starting from the next frame of the cleared image.

7. The outdoor equipment detection data collection method according to claim 5, characterized in that: The location information includes the center point coordinates of the detection area and its width and height data.

8. Outdoor equipment detection data acquisition system, characterized by: include: Visual equipment, used to collect video data of the target detection object within the visible area; An identification module, used to identify a detection area of ​​each frame of the video data and obtain location information of the detection area, wherein the detection area is an area where the target detection object is displayed in the current image; The buffer area is used to periodically pre-store each frame of the video data, and output the intermediate image and its position information to the sample collection module and clear the data after processing the current pre-stored image each time, and then continue to pre-store the next batch of images; the preset number is an odd number, and the frame of image located in the middle position of the buffer area is the intermediate image of the current buffer area; A logic control module determines whether a first detection error and a second detection error occur in an intermediate image in a buffer area, wherein the first detection error refers to that the intermediate image in the current buffer area fails to identify a detection area, and the detection area is identified in other images before or after the intermediate image in the buffer area; the second detection error refers to that the intermediate image in the current buffer area identifies a detection area, and the detection area is not identified in other images in the buffer area; if a first detection error occurs in an intermediate image in the current buffer area, the image of the most recent frame that identifies the detection area is found before and after the intermediate image, respectively, and the position information of the detection area in the two frames of images is obtained respectively, and the position information of the detection area of ​​the current intermediate image is calculated by smoothing the two position information; A sample collection module, including a positive sample set and a negative sample set, wherein the positive sample set is used to collect intermediate images in the buffer area where a first detection error occurs and the detection area position information obtained after calculation, and the negative sample set is used to collect intermediate images in the buffer area where a second detection error occurs and the position information thereof; The positive sample set and the negative sample set respectively transmit their data to the data center and then clear the data.

9. Outdoor equipment detection data acquisition device, characterized by: It includes the outdoor equipment detection data acquisition system as described in claim 8.

10. A storage medium, characterized in that: It includes the outdoor equipment detection data acquisition system as described in claim 8.

Images