Construction status identification method, device, processing equipment and storage medium

Abstract

The present application relates to a construction status recognition method, device, processing equipment and storage medium, the method comprising: acquiring an image sequence of an area to be detected, the image sequence including multiple continuous image frames; performing target tracking on at least one candidate construction object in the image sequence, respectively determining the statistical number of occurrences of the at least one candidate construction object in the multiple continuous image frames, the statistical number being obtained according to a preset statistical rule; determining a candidate construction object whose statistical number meets a preset condition as a construction object; and determining that the area to be detected is in a construction state when the appearance duration of the construction object is greater than a preset duration threshold. The construction status recognition method provided in the embodiment of the present application can reduce the probability of misjudgment and missed judgment caused by target occlusion or detector instability, and improve the recognition accuracy of the construction status recognition method.

Description

Technical Field

[0001] The present application relates to the field of video surveillance, and in particular to a construction status identification method, device, processing equipment and storage medium. Background Art

[0002] With the continuous development of artificial intelligence and intelligent monitoring technology, using intelligent monitoring technology to solve road traffic problems and reduce traffic accidents has become a new trend.

[0003] In the related technology, intelligent monitoring equipment can be used to collect image data or video data of roads, and then analyze the image data or video data to identify the construction status and road conditions of urban roads, such as the judgment result that the road is under construction or the road is congested due to road construction, etc., so as to provide real-time traffic information for travelers. However, in the process of collecting information, the related technology is prone to problems such as vehicles blocking construction signs, which reduces the accuracy of construction status recognition.

[0004] Therefore, the relevant technology urgently needs a high-accuracy construction status identification method. Summary of the invention

[0005] Based on this, in response to the above-mentioned technical problems, the present application provides a construction status identification method, device, processing equipment and storage medium, so as to at least solve the problem in the related technology that vehicles easily block construction signs during the collection process, thereby reducing the accuracy of construction status identification.

[0006] In a first aspect, an embodiment of the present application provides a construction status identification method, the method comprising:

[0007] Acquire an image sequence of the area to be detected, wherein the image sequence includes a plurality of continuous image frames;

[0008] Tracking at least one candidate construction object in the image sequence, and determining the statistical number of occurrences of the at least one candidate construction object in the plurality of consecutive image frames, wherein the statistical number is obtained according to a preset statistical rule;

[0009] Determine the candidate construction object whose statistical number of times meets the preset condition as the construction object;

[0010] When the appearance time of the construction object is greater than a preset time threshold, it is determined that the area to be detected is in a construction state.

[0011] Optionally, in one embodiment of the present application, the preset statistical rules include:

[0012] Setting a plurality of data sets for storing the identification information of the candidate construction object, wherein the plurality of data sets respectively correspond to different construction object confidences, and the construction object confidences are used to indicate the probability that the candidate construction object is determined to be a construction object;

[0013] According to the time sequence of the plurality of consecutive image frames, dynamically adjusting the data set to which the at least one candidate construction object is added according to the number of occurrences and the manner in which the candidate construction object appears;

[0014] The number of candidate construction object identification information in the target data set is counted, and the number is used as the statistical number, and the target data set includes a data set whose construction object confidence is greater than a preset confidence threshold.

[0015] Optionally, in one embodiment of the present application, the preset statistical rules include:

[0016] According to the time sequence of the plurality of consecutive image frames, the number of occurrences of the at least one candidate construction object in the consecutive image frames is accumulated respectively to dynamically generate the statistical number;

[0017] In the case where the statistical number is greater than zero and the candidate construction object does not appear in the continuous image frames, subtracting the number of non-appearances from the statistical number;

[0018] When the statistical number is less than or equal to a preset threshold, the accumulation of the statistical number is stopped.

[0019] Optionally, in one embodiment of the present application, determining the candidate construction object whose statistical number of times meets a preset condition as the construction object includes:

[0020] The candidate construction object whose statistical number is greater than a preset statistical number threshold is determined as the construction object.

[0021] Optionally, in an embodiment of the present application, when the appearance duration of the construction object is greater than a preset duration threshold, after determining that the detection area is in a construction state, the method further includes:

[0022] Acquiring location information of the construction object;

[0023] The construction area of ​​the area to be detected is determined according to the location information.

[0024] Optionally, in one embodiment of the present application, the target tracking of at least one candidate construction object in the image sequence includes:

[0025] Inputting the plurality of continuous image frames into a construction object detection model respectively, and outputting position information and identification information of at least one candidate construction object in the image frame through the construction object detection model;

[0026] The at least one candidate construction object is tracked according to the identification information.

[0027] In a second aspect, an embodiment of the present application further provides a construction status identification device, the device comprising:

[0028] An acquisition module, used for acquiring an image sequence of the area to be detected, wherein the image sequence includes a plurality of continuous image frames;

[0029] A statistical module, used for tracking at least one candidate construction object in the image sequence, and determining the statistical number of occurrences of the at least one candidate construction object in the plurality of consecutive image frames, wherein the statistical number is obtained according to a preset statistical rule;

[0030] A first determination module, configured to determine the candidate construction objects whose occurrence times meet a preset condition as construction objects;

[0031] The second determination module determines that the detection area is in a construction state when the appearance duration of the construction object is greater than a preset duration threshold.

[0032] In a third aspect, an embodiment of the present application further provides a processing device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned construction status identification method when executing the computer program.

[0033] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having computer program instructions stored thereon, and the computer program instructions, when executed by a processor, implement the steps of the above-mentioned construction status identification method.

[0034] In a fifth aspect, an embodiment of the present application further provides a chip, comprising at least one processor, wherein the processor is used to run computer program instructions stored in a memory to execute the steps of the construction status identification method described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0037] Figure 2A method flow chart of a construction status identification method provided by an embodiment of the present application;

[0038] Figure 3 This is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0039] Figure 4 1 is a schematic diagram of the module structure of the construction status identification device 103 provided in an embodiment of the present application;

[0040] Figure 5 It is a schematic diagram of the module structure of the processing device 500 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means, and should not be understood as insufficient contents disclosed in the present application.

[0042] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0043] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to greater than or equal to two. The terms "first", "second", "third" and the like involved in this application are merely to distinguish similar objects and do not represent a specific ordering of objects.

[0044] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, devices, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0045] In order to facilitate understanding of the embodiments of the present application, the following first describes the structure of one of the construction status identification systems based on the embodiments of the present application. Figure 1 , Figure 1 It is a structural diagram of a construction status identification system provided in an embodiment of the present application, the system comprising a collection device 101 and a construction status identification device 103, wherein the collection device 101 and the construction status identification device 103 can communicate via a network to send a collected image sequence for construction status identification to the construction status identification device 103, and the construction status identification device 103 completes the identification of the construction status of the area to be detected.

[0046] The acquisition device 101 may be an electronic device with image acquisition and data transmission capabilities. For example, the acquisition device 101 may include a camera, a laser radar, etc. Among them, the camera may include a gun-type camera, a pan-tilt camera, an infrared camera, a binocular camera, a depth camera, etc., and the laser radar may include a single-line radar, a multi-line radar, etc., and the present application does not limit the acquisition device 101. The acquisition device 101 may also include a roadside unit, and the roadside unit may be installed on a road section with a construction status identification requirement. It should be noted that a roadside unit may be used to collect an image sequence of the area to be detected, or multiple roadside units may cooperate to collect an image sequence of the area to be detected. Among them, the data transmission function of the acquisition device 101 may be realized by a high-gain directional beam-controlled read-write antenna and a radio frequency controller. The high-gain directional beam-controlled read-write antenna is a microwave transceiver module, which is responsible for sending / receiving, modulating / demodulating, encoding / decoding, encrypting / decrypting signals and data; the radio frequency controller is a module that controls the transmission and reception of data and processes the information sent and received to the upper computer.

[0047] The construction status recognition device 103 can be an electronic device with data processing capability and data receiving and sending capability, and can be a physical device such as a host, a rack server, a blade server, etc., or a virtual device such as a virtual machine, a container, etc. After acquiring the image sequence of the area to be detected, the construction status recognition device 103 can track at least one candidate construction object in the image sequence, and respectively determine the statistical number of occurrences of the at least one candidate construction object in the multiple continuous image frames, and the statistical number is obtained according to a preset statistical rule; then, the candidate construction object whose statistical number meets the preset conditions can be determined as a construction object. Finally, when the appearance time of the construction object is greater than the preset time threshold, it is determined that the area to be detected is in a construction state. In addition to electronic devices, the construction status recognition device 103 can also include various forms such as non-volatile computer-readable storage media, computer program products, chips, etc. As a non-volatile computer-readable storage medium, a computer program product, and a chip, the construction status recognition device 103 can be coupled to the inside of the acquisition device 101, so that the acquisition device 101 has the function of identifying the construction state of the area to be detected.

[0048] The construction status identification method described in this application is described in detail below with reference to the accompanying drawings. Figure 2It is a flow chart of an embodiment of the construction status identification method provided by the present application. Although the present application provides the method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiments of the present application. During the actual construction status identification process or when the method is executed, the method can be executed in the method sequence shown in the embodiments or drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0049] Specifically, an implementation example of the construction status identification method provided by the present application is as follows: Figure 2 As shown, the method may include:

[0050] S201: Acquire an image sequence of a region to be detected, wherein the image sequence includes a plurality of continuous image frames.

[0051] In an embodiment of the present application, the area to be detected may include an area with a construction status identification requirement. The area may include, for example, a road area, a park, a community, and other place areas. The acquisition device 101 may be used to acquire images or videos of the area to be detected. Depending on the acquisition principle of the acquisition device 101, the types of images acquired are also different. Optionally, in an embodiment of the present application, the acquisition device 101 may include an infrared camera, and the type of image acquired is an infrared thermal imaging image. During road construction, there may be construction at night, and clear images can be captured even at night using an infrared camera. Of course, in other embodiments, the image type may also include visible light images, three-dimensional point cloud images, depth images, and the like, which are not limited in this application. The image may include any format such as BMP, JPEG, PNG, SVG, etc. The image sequence may include multiple continuous image frames, and the multiple continuous image frames may include continuous multi-frame images acquired at a preset frame rate, and the preset frame rate may be set to, for example, 1fps, 5fps, 10fps, etc., which are not limited here.

[0052] It should be noted that the process of acquiring the image sequence in the embodiment of the present application is a continuous process, and the image sequence can be acquired once every preset time interval. For example, the acquisition device 101 can be set to upload the image or video acquired in the previous minute to the construction status recognition device 103 every minute, and the uploaded image or video is processed by the construction status recognition device 103. Of course, the preset time interval can also be set to 20s, 40s, 2min, etc. according to actual needs, and there is no limitation here.

[0053] S203: Tracking at least one candidate construction object in the image sequence, and determining the statistical number of occurrences of the at least one candidate construction object in the plurality of consecutive image frames, wherein the statistical number is obtained according to a preset statistical rule.

[0054] In an embodiment of the present application, the at least one candidate construction object may include a target object in the image sequence that needs to be determined whether it is a construction object, and the construction object may include a construction vehicle, a construction warning sign, a road cone, a construction fence, etc. In an embodiment of the present application, after acquiring the image sequence of the area to be detected, target tracking may be performed on at least one candidate construction object in the image sequence, and the statistical number of times the at least one candidate construction object appears in the multiple consecutive image frames may be determined respectively. Among them, the target tracking may include tracking the same candidate construction object in different image frames, and determining the number of times the candidate construction object appears in the multiple consecutive image frames. In one embodiment of the present application, a machine learning model may be used to perform target detection on the multiple consecutive image frames respectively. Specifically, the target tracking of at least one candidate construction object in the image sequence may include:

[0055] S301: inputting the plurality of continuous image frames into a construction object detection model respectively, and outputting position information and identification information of at least one candidate construction object in the image frame through the construction object detection model;

[0056] S303: Tracking the at least one candidate construction object according to the identification information.

[0057] In the embodiment of the present application, the construction object detection model may include a model trained by deep learning, and is used to detect a regional image of at least one candidate construction object in an image frame. Figure 3 As shown, Figure 3 The image frame shown is input into the construction object detection model, and the construction object detection model can identify the regional image of at least one candidate construction object in the image frame, and can also use a bounding box to frame the regional image of the at least one candidate construction object. The position information may include the position of the regional image in the image frame. In addition, the construction object detection model can also distinguish different candidate construction objects and set identification information for the detected candidate construction objects, and the identification information is used to characterize the uniqueness of the candidate construction object. Figure 3 As shown, the location information and identification information “Warning Sign 1 ” of the at least one construction object may be marked next to the boundary box. Then, the number of occurrences of the candidate construction object in the plurality of consecutive image frames may be determined based on the identification information.

[0058] It should be noted that the construction object detection model can be trained by a variety of target detection network frameworks. Typically, the target detection network frameworks include AlexNet, ResNet, ResNet1001 (pre-activation), Hourglass, Inception, Xception, SENet, etc. This application does not limit this.

[0059] In the embodiment of the present application, the at least one candidate construction object in the image sequence can be identified using a machine learning model. Since the machine learning model is learned based on a large number of samples, this recognition method has high recognition accuracy and efficiency.

[0060] In an embodiment of the present application, taking into account the possibility of target occlusion or inaccurate detection, in order to improve the accuracy of counting the number of occurrences, the number of occurrences can be counted using preset statistical rules to obtain the statistical number. The preset statistical rules may include rules for distinguishing the number of occurrences. The rules may, for example, include not counting the number of occurrences that do not meet the conditions. In some other embodiments of the present application, the preset statistical rules may include filtering rules and statistical rules. Among them, the filtering rules are used to indicate the rules for not counting the number of occurrences, and the statistical rules are used to indicate the rules for counting the number of occurrences.

[0061] In one embodiment of the present application, the preset statistical rules may include:

[0062] S401: setting a plurality of data sets for storing identification information of the candidate construction object, wherein the plurality of data sets respectively correspond to different construction object confidences, and the construction object confidences are used to indicate the probability that the candidate construction object is determined to be a construction object;

[0063] S403: dynamically adjusting the data set to which the at least one candidate construction object is added according to the time sequence of the plurality of consecutive image frames and the number of occurrences and the manner in which the candidate construction object appears;

[0064] S405: Counting the number of candidate construction object identification information in a target data set, and taking the number as the statistical number, the target data set includes a data set whose construction object confidence is greater than a preset confidence threshold.

[0065] In the embodiment of the present application, according to the preset statistical rule, the data set is set to store the identification information of the candidate construction object. The function of the data set is to organize at least one data element of the same type together, and the data element may include the identification information of the candidate construction object. In the embodiment of the present application, different data sets may correspond to different construction object confidences, that is, the identification information of the at least one candidate construction object is divided into multiple data sets according to the construction object confidence. The construction object confidence is used to indicate the probability that the candidate construction object is determined to be a construction object, that is, the higher the construction object confidence, the higher the probability that the candidate construction object in the corresponding data set is determined to be a construction object. In one example, multiple data sets such as "construction object candidate set", "construction object confirmation set" and "construction object waiting set" may be set, wherein the construction object confidence of the "construction object confirmation set" is the highest, and the construction confidence of the "construction object candidate set" is the lowest. Of course, in other embodiments, multiple data sets such as "data set 1", "data set 2", "data set 3" and "data set 4" may also be set, and the present application does not limit the number and name of the data sets.

[0066] In the embodiment of the present application, the data set can be set as a variable, that is, the identification information of the candidate construction object in the data set can be changed. In other words, the identification information of the candidate construction object can be stored in different data sets over time, but the identification information of each candidate construction object is generally stored in only one data set. Specifically, in the embodiment of the present application, the data set to which the at least one candidate construction object is added can be dynamically adjusted according to the time sequence of the multiple consecutive image frames and the number of occurrences and the manner of occurrence of the candidate construction object. For example, the multiple consecutive image frames can be processed in time sequence, and each time an image frame is processed, the data set to which the at least one candidate construction object in the image frame belongs is determined. Specifically, the data set to which the candidate construction object belongs can be determined according to the number of occurrences and the manner of occurrence of the candidate construction object, and the number of occurrences can include the cumulative number of occurrences of the candidate construction object as of the current image frame.

[0067] The following is a description of the process of dynamically adjusting the multiple data sets in conjunction with the specific examples shown in Table 1. For example, for multiple data sets such as "construction object candidate set", "construction object confirmation set", and "construction object waiting set", the type of identification information of the candidate construction objects stored in each data set can be defined first. For example, the "construction object candidate set" can be used to store the identification information of the candidate construction objects detected once from the image sequence, the "construction object confirmation set" can be used to store the identification information of the candidate construction objects that have been continuously detected from the image sequence for a number of times greater than a preset threshold, and the "construction object waiting set" can be used to store the identification information of the candidate construction objects transferred from the "construction object confirmation set", specifically referring to the identification information of the candidate construction objects that appear non-continuously. As shown in Table 1, for an image sequence with 4 image frames, candidate construction objects such as warning sign 1, warning sign 2, and road cone 1 are first detected from image frame 1, and the identification information of these candidate construction objects is stored in the "construction object candidate set". In image frame 2, warning sign 1, road cone 1, road cone 2 and road cone 3 are detected. Assuming that a candidate construction object is detected twice or more continuously, its identification information needs to be stored in the "construction object confirmation set". Based on this, the identification information of warning sign 1, road cone 1, road cone 2 and road cone 3 can be transferred to the "construction object confirmation set". In image frame 3, warning sign 1, road cone 1 and road cone 3 are detected, but road cone 2 is not detected. At this time, road cone 2 appears non-continuously. Therefore, road cone 2 needs to be transferred to the "construction object waiting set". For the candidate construction objects in the "construction object waiting set", it can be set that if the number of consecutive non-appearances of the candidate construction object is greater than or equal to two times, the identification information of the candidate construction object is removed from the "construction object waiting set". Based on this, in image frame 4, road cone 2 is not detected twice in a row, so road cone 2 is removed from the "construction object waiting set".

[0068] Table 1 Schematic diagram of dynamic adjustment of data set

[0069]

[0070] Finally, the number of candidate construction object identification information in the target data set can be counted and the number can be used as the statistical number. The target data set includes the data set whose construction object confidence is greater than the preset confidence threshold. For example, in the above example, the target data set can include a "construction object confirmation set", and can also include a "construction object confirmation set" and a "construction object waiting set". Then, in the above example, the number of candidate construction object identification information in the target data set is 3, that is, the statistical number is 3.

[0071] In this embodiment, the state of the at least one candidate construction object is dynamically adjusted using the data set. The processing method is simple and accurate, and the possibility of misjudging the candidate construction object is reduced.

[0072] In another embodiment of the present application, the preset statistical rules include:

[0073] S501: Accumulating the number of occurrences of the at least one candidate construction object in the consecutive image frames according to the time sequence of the multiple consecutive image frames, and dynamically generating the statistical number;

[0074] S503: when the statistical number is greater than zero and the candidate construction object does not appear in the continuous image frames, subtract the number of times that the candidate construction object does not appear from the statistical number;

[0075] S505: When the statistical number is less than or equal to a preset threshold, stop accumulating the statistical number.

[0076] In an embodiment of the present application, the at least one candidate construction object may appear in the continuous image frames in a continuous manner or in an intermittent manner. The continuous appearance means that the candidate construction object appears in the continuous image frames, and the intermittent appearance means that the candidate construction object appears in the continuous image frames intermittently. Specifically, in one example, the image sequence includes image frames 1, 2, 3, and 4. Warning sign 1, road cone 1, road cone 2, and road cone 3 are detected in image frame 1 and image frame 2. Warning sign 1, road cone 1, and road cone 2 are detected in image frame 3, but road cone 3 is not detected. Thus, the statistical number of road cone 3 dynamically generated is 2 times. In addition, since the appearance of road cone 3 is intermittent, the number of times that road cone 3 does not appear is 1, and the number of times that road cone 3 does not appear is subtracted from the statistical number of 2 times, and the statistical number is dynamically adjusted to 1 time. In the case that road cone 3 is still not detected in image frame 4, the statistical number is adjusted to 0 times, and the accumulation of the statistical number of road cone 3 is stopped.

[0077] Through the above embodiments, the statistical times can be dynamically adjusted according to preset rules to achieve the association of multiple image frames in the image sequence, thereby reducing the probability of misjudgment and missed judgment caused by target occlusion or detector instability.

[0078] S207: Determine the candidate construction object whose statistical times meet the preset conditions as the construction object.

[0079] In an embodiment of the present application, after determining the statistical number of at least one candidate construction object in the above manner, it is possible to determine whether the candidate construction object is a construction object based on the relationship between the statistical number of the candidate construction object and the preset condition. In the case where it is determined that the statistical number of the candidate construction object meets the preset condition, the candidate construction object is determined to be a construction object. The preset condition can be set by the user according to actual needs, or it can be determined based on the relationship between the statistical number and the reference statistical parameter. Specifically, in an embodiment of the present application, determining the candidate construction object whose statistical number meets the preset condition as a construction object includes: determining the candidate construction object whose statistical number is greater than a preset statistical number threshold as a construction object. In an embodiment of the present application, a preset statistical number threshold can be pre-set, for example, it can be set to 1 time, 2 times, 3 times, etc. In the case where the statistical number is greater than the preset statistical number threshold, the candidate construction object is determined to be a construction object.

[0080] S209: When the appearance duration of the construction object is greater than a preset duration threshold, determining that the area to be detected is in a construction state.

[0081] In an embodiment of the present application, the duration of the appearance of the construction object can be determined according to the statistical time when the at least one construction object appears in the image sequence. Specifically, in one example, when it is determined that the warning sign 1 appears continuously in the image frame 1-image frame 3 of the image sequence, the statistical number of the warning sign 1 can be determined to be 3 times according to the preset rule. When the preset frame rate of the acquisition setting 101 is 1fps, the time interval from the image frame 1 to the image frame 3 can be determined to be 3s, thereby determining that the duration of the appearance of the warning sign 1 is 3s. It should be noted that, when there are multiple construction objects, the duration of the appearance of all construction objects can be counted as the duration of the appearance of the construction object. After determining the duration of the appearance, it can be determined whether the area to be detected is in a construction state according to the comparison result between the preset duration and the preset duration threshold. Specifically, in an embodiment of the present application, when the preset duration is greater than the preset duration threshold, the area to be detected is determined to be in a construction state. For example, when the duration of the appearance of 50s is greater than the preset duration threshold of 40s, the area to be detected is determined to be in a construction state.

[0082] The construction status recognition method provided by each embodiment of the present application can track the at least one candidate construction object in the to-be-detected area based on the image sequence of the to-be-detected area, and respectively determine the statistical number of times the at least one candidate construction object appears in the multiple consecutive image frames. Then, the statistical number can be dynamically adjusted according to preset rules. This achieves the association of multiple frames of images in the image sequence, reduces the probability of misjudgment and missed judgment caused by target occlusion or detector instability, and thus improves the recognition accuracy of the construction status recognition method.

[0083] After determining that the area to be detected is under construction, the construction area in the area to be detected can be determined according to the location information of the construction object. Specifically, in one embodiment of the present application, when the appearance duration of the construction object is greater than a preset duration threshold, after determining that the detection area is under construction, the following further includes:

[0084] S601: Acquire the location information of the construction object;

[0085] S603: Determine the construction area of ​​the area to be detected according to the location information.

[0086] In an embodiment of the present application, since the construction object detection model can output the location information of the at least one candidate construction object. The location information may include specific coordinate values, and may also include a location area. After determining the construction object, the location information corresponding to the construction object may be determined based on the output result of the construction object detection model. Then, the construction area of ​​the area to be detected may be determined based on the location information. In an embodiment of the present application, the construction area may be determined based on the edgemost location information in the location information, or the construction area may be determined using a convex hull calculation method. Specifically, it may include determining, based on the location information, the outermost convex hull that can surround the location information, and determining the construction area based on the convex hull.

[0087] On the other hand, the present application also provides a construction status identification device 103, such as Figure 4 As shown, the construction status identification device 103 includes:

[0088] An acquisition module 1031 is used to acquire an image sequence of the area to be detected, wherein the image sequence includes a plurality of continuous image frames;

[0089] A statistical module 1033 is used to track at least one candidate construction object in the image sequence, and determine the statistical number of occurrences of the at least one candidate construction object in the plurality of consecutive image frames, wherein the statistical number is obtained according to a preset statistical rule;

[0090] A first determination module 1035 is used to determine the candidate construction object whose occurrence times meet the preset conditions as the construction object;

[0091] The second determination module 1037 determines that the detection area is in a construction state when the appearance duration of the construction object is greater than a preset duration threshold.

[0092] On the other hand, the present application further provides a processing device, including a memory and a processor, wherein the memory stores computer program instructions, and the processor is configured to run the computer program instructions to execute the construction status identification method described in each of the above embodiments.

[0093] The processing device may be a physical device or a cluster of physical devices, or a virtualized cloud device, such as at least one cloud computing device in a cloud computing cluster. For ease of understanding, the present application uses the processing device as an independent physical device to illustrate the structure of the processing device.

[0094] like Figure 5 As shown, the processing device 500 includes: a processor and a memory for storing processor computer program instructions; wherein the processor is configured to implement the above-mentioned device when executing the computer program instructions. The processing device 500 includes a memory 501, a processor 503, a bus 505 and a communication interface 507. The memory 501, the processor 503 and the communication interface 507 communicate through the bus 505. The bus 505 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The communication interface 507 is used for communicating with the outside.

[0095] The processor 503 may be a central processing unit (CPU). The memory 501 may include a volatile memory, such as a random access memory (RAM). The memory 501 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a HDD or a SSD, etc.

[0096] Those skilled in the art will understand that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0097] On the other hand, the present application further provides a chip, comprising at least one processor, wherein the processor is used to run computer program instructions stored in a memory to execute the steps of the construction status identification method described in the above-mentioned embodiments.

[0098] On the other hand, the present application further provides a computer-readable storage medium having computer program instructions stored thereon, and the computer program instructions, when executed by a processor, implement the steps of the above-mentioned construction status identification method.

[0099] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (Random Access Memory, RAM), a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (Electrically Programmable Read-Only-Memory, EPROM or flash memory), a static random access memory (Static Random-Access Memory, SRAM), a portable compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), a digital versatile disc (Digital Video Disc, DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0100] The computer program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer program instructions from the network and forwards the computer program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0101] The computer program instructions for performing the operation of the present application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Small talk, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The computer program instructions may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on the remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, by utilizing the status information of computer program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA), the electronic circuit can execute the computer program instructions to implement various aspects of the present application.

[0102] Various aspects of the present application are described herein with reference to the flowcharts and / or block diagrams of the methods and devices according to the embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer program instructions.

[0103] These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0104] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0105] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, system and method according to multiple embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a different order from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, depending on the functions involved.

[0106] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A construction status identification method, characterized in that: The method comprises: Acquire an image sequence of the area to be detected, wherein the image sequence includes a plurality of continuous image frames; Tracking at least one candidate construction object in the image sequence, and determining the statistical number of occurrences of the at least one candidate construction object in the plurality of consecutive image frames, wherein the statistical number is obtained according to a preset statistical rule; Determine the candidate construction object whose statistical number of times meets the preset condition as the construction object; When the appearance time of the construction object is greater than a preset time threshold, determining that the area to be detected is in a construction state; The preset statistical rules include: Setting a plurality of data sets for storing the identification information of the candidate construction object, wherein the plurality of data sets respectively correspond to different construction object confidences, and the construction object confidences are used to indicate the probability that the candidate construction object is determined to be a construction object; According to the time sequence of the plurality of consecutive image frames, dynamically adjusting the data set to which the at least one candidate construction object is added according to the number of occurrences and the manner in which the candidate construction object appears; The number of candidate construction object identification information in the target data set is counted, and the number is used as the statistical number, and the target data set includes a data set whose construction object confidence is greater than a preset confidence threshold.

2. The method according to claim 1, characterized in that The preset statistical rules include: According to the time sequence of the plurality of consecutive image frames, the number of occurrences of the at least one candidate construction object in the consecutive image frames is accumulated respectively to dynamically generate the statistical number; In the case where the statistical number is greater than zero and the candidate construction object does not appear in the continuous image frames, subtracting the number of non-appearances from the statistical number; When the statistical number is less than or equal to a preset threshold, the accumulation of the statistical number is stopped.

3. The method according to claim 1, characterized in that The step of determining the candidate construction object whose statistical number of times satisfies a preset condition as the construction object comprises: The candidate construction object whose statistical number is greater than a preset statistical number threshold is determined as the construction object.

4. The method according to claim 1, characterized in that: After determining that the detection area is in a construction state when the appearance duration of the construction object is greater than a preset duration threshold, the method further includes: Acquiring location information of the construction object; The construction area of ​​the area to be detected is determined according to the location information.

5. The method according to claim 1, characterized in that The target tracking of at least one candidate construction object in the image sequence includes: Inputting the plurality of continuous image frames into a construction object detection model respectively, and outputting position information and identification information of at least one candidate construction object in the image frame through the construction object detection model; The at least one candidate construction object is tracked according to the identification information.

6. A construction status identification device, characterized in that: The device comprises: An acquisition module, used for acquiring an image sequence of the area to be detected, wherein the image sequence includes a plurality of continuous image frames; A statistical module, used for tracking at least one candidate construction object in the image sequence, and determining the statistical number of occurrences of the at least one candidate construction object in the plurality of consecutive image frames, wherein the statistical number is obtained according to a preset statistical rule; A first determination module, configured to determine the candidate construction object whose statistical number of times meets a preset condition as a construction object; A second determination module, when the appearance time of the construction object is greater than a preset time threshold, determines that the detection area is in a construction state; The preset statistical rules include: Setting a plurality of data sets for storing the identification information of the candidate construction object, wherein the plurality of data sets respectively correspond to different construction object confidences, and the construction object confidences are used to indicate the probability that the candidate construction object is determined to be a construction object; According to the time sequence of the plurality of consecutive image frames, dynamically adjusting the data set to which the at least one candidate construction object is added according to the number of occurrences and the manner in which the candidate construction object appears; The number of candidate construction object identification information in the target data set is counted, and the number is used as the statistical number, and the target data set includes a data set whose construction object confidence is greater than a preset confidence threshold.

7. A processing device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the construction status recognition method described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the construction status identification method described in any one of claims 1 to 5 are implemented.

9. A chip, characterized in that: The system comprises at least one processor, wherein the processor is used to run computer program instructions stored in a memory to execute the steps of the method for identifying the construction status according to any one of claims 1 to 5.

Images