A method, medium and system for detecting the absence of road signs and security facilities

Abstract

The application discloses a kind of road sign and the detection method, medium and system of security and protection facilities loss, comprising: the database of road target detection object is established;In the process of detecting vehicle driving, every driving preset distance, through the camera installed on the detection vehicle to collect an image of the road in front of the detection vehicle;Identify each target detection object in all collected images, and record the position identification result and the kind identification result of each target detection object, obtain the first sequence of each target detection object by position identification result;According to the kind identification result, the first sequence of each target detection object is corrected, and the second sequence of each target detection object is obtained;For each target detection object, the second sequence of the target detection object is compared with the standard sequence of the target detection object in the database, to determine whether the target detection object is missing.The application can automatically realize the missing detection of sign and security and protection facilities, reduce manual intervention, and improve detection and identification efficiency.

Description

Technical Field

[0001] This invention relates to the field of road inspection technology, and in particular to a method, medium and system for detecting missing road signs and safety facilities. Background Technology

[0002] Based on the transportation construction policy of emphasizing both construction and maintenance, the concept of full asset management for highways has been proposed, leading to increased emphasis on the inspection of road safety protection facilities and signs by various maintenance and management units. To meet the management needs of maintenance units, current methods utilize front-facing cameras on inspection vehicles to collect images and perform frame-by-frame recognition and comparison. While computer vision-based inspection has been implemented, accelerating the initial comprehensive inspection and labeling of highway facilities, subsequent sign classification, location information, and missing sign detection still require significant manual comparison and marking work. Automatic missing sign labeling and detection cannot be achieved, which greatly limits the progress of consulting services. Furthermore, current technology does not enable mutual inspection of safety engineering and warning signs. Summary of the Invention

[0003] This invention provides a method, medium, and system for detecting missing road signs and security facilities, to solve the problem of low efficiency caused by manual detection of missing road signs and security facilities in existing technologies.

[0004] Firstly, a method for detecting missing road signs and safety facilities is provided, including:

[0005] A database of road target detection objects is established, wherein the types of target detection objects include: signs and security facilities, and the database includes: a standard sequence of the geographical location of each target detection object and the category of each target detection object;

[0006] During the operation of the inspection vehicle, an image of the road ahead of the inspection vehicle is captured by a camera installed on the vehicle every preset distance.

[0007] In all the acquired images, each of the target detection objects is identified, and the location identification result and type identification result of each target detection object are recorded to obtain a first sequence of each target detection object composed of the location identification results;

[0008] Based on the category identification results, the first sequence of each target detection object is modified to obtain the second sequence of each target detection object;

[0009] For each target detection object, the second sequence of the target detection object is compared with the standard sequence of the target detection object in the database to determine whether the target detection object is missing.

[0010] In this sequence, each element in the standard sequence, the first sequence, and the second sequence takes the value of 0 or 1. Each element corresponds to a geographical location, and each element is arranged in order of geographical location. If the value of an element is 0, it means that the geographical location does not have the target object. If the value of an element is 1, it means that the geographical location has the target object.

[0011] In a second aspect, a computer-readable storage medium is provided, wherein computer program instructions are stored on the computer-readable storage medium; when the computer program instructions are executed by a processor, they implement the method for detecting missing road signs and security facilities as described in the first aspect embodiment above.

[0012] Thirdly, a detection system for missing road signs and security facilities is provided, comprising: a computer-readable storage medium as described in the second aspect embodiment above.

[0013] Thus, this embodiment of the invention can realize automatic detection of missing signs and security facilities, and can perform mutual inspection of signs and security facilities, reducing manual intervention and improving detection and identification efficiency. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of a preferred embodiment of the method for detecting missing road signs and security facilities according to the present invention;

[0016] Figure 2 This is a flowchart of a method for detecting missing road signs and security facilities according to another preferred embodiment of the present invention;

[0017] Figure 3 This is a schematic diagram of the backtracking algorithm model according to an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] This invention discloses a method for detecting missing road signs and safety facilities. For example... Figure 1As shown, the method includes the following steps:

[0020] Step S101: Establish a database of road target detection objects.

[0021] For each road, a database of target objects on that road can be established. By establishing the database and storing the relevant information in a structured manner, data query and matching functions can be provided for retrospective detection in subsequent steps.

[0022] Specifically, the types of objects to be detected include: signs and security facilities. Further, the types of signs include: warning signs, prohibition signs, and instruction signs. Further, security facilities include: crash barriers, speed bumps, etc.

[0023] Specifically, the database includes: a standard sequence of the geographical location of each target and the category of each target.

[0024] In this standard sequence, each element has a value of 0 or 1, and each element corresponds to a geographical location. The geographical location can be a station number or GPS data. Each element is arranged in geographical location order. For example, the database can record road routes and directions, and elements of different geographical locations can be arranged sequentially according to these routes and directions. The element's sequence number can be represented by an index. In this embodiment of the invention, in a sequence, the first sort corresponds to index 0, the second sort corresponds to index 1, the third sort corresponds to index 2, and so on. If the element is 0, it indicates that the geographical location does not have the target object; if the element is 1, it indicates that the geographical location has the target object. For example, if the road has station numbers 1-5, and the target object is present at station number 4, then the standard sequence is 00010.

[0025] Step S102: During the driving of the inspection vehicle, every preset distance traveled, an image of the road in front of the inspection vehicle is captured by the camera installed on the inspection vehicle.

[0026] The camera is typically mounted on the top of the inspection vehicle. The preset distance can be determined based on the actual situation. For example, a preset distance of 10m means that the inspection vehicle will take an image every 10m as it travels on the road.

[0027] Step S103: Identify each target detection object in all acquired images, and record the location identification result and category identification result of each target detection object to obtain the first sequence of each target detection object composed of the location identification result.

[0028] Specifically, image recognition technology can employ existing techniques, such as using computer vision recognition technology to construct a suitable deep neural network model, and training and optimizing the model using random data from a preceding image database to identify target objects, their types, and geographical locations. For example, through...<Deep Learning for Large-Scale Traffic-Sign Detection and Recognition> Domen Tabernik and Danijel Skocaj, or,<MR-CNN:A Multi-Scale Region-Based Convolutional Neural Network for Small TrafficSign Recognition> ZHIGANG LIU, JUAN DU, FENG TIAN and JIAZHENG WEN mentioned the technology used for image recognition. Obtaining the geographical location from the image can be achieved by acquiring station numbers or GPS data, or by transforming the image coordinate system and road coordinate system using existing technologies.

[0029] The representation of the first sequence is consistent with the target sequence, and will not be repeated here. For example, if the road is marked with chainages 1 to 5, and the target object is identified at chainage 4, then the first sequence is 00010.

[0030] Step S104: Based on the category identification results, the first sequence of each target detection object is corrected to obtain the second sequence of each target detection object.

[0031] Typically, camera lenses vary in focal length, resulting in different viewing angles and blind spot areas. Furthermore, the lane in which the inspection vehicle travels also creates different blind spots. Therefore, at locations marked consecutively, the actual geographical location (station number or GPS) may not be accurate. This step is used to correct the geographical location.

[0032] Specifically, the process is as follows:

[0033] (1) For each target detection object, the following is adopted: Calculate the shift number of the first sequence of the target detection object.

[0034] Among them, P T Indicates the number of shifts. c represents the distance between the center line of the lane the detection vehicle is traveling on and the slope or edge line, θ represents the viewing angle of the camera, and H represents the image acquisition frequency, which is equivalent to the preset distance / time.

[0035] The camera's viewing angle depends on the device's actual parameters, as shown in Table 1. When applying the camera, select the viewing angle from Table 1 based on its actual parameters.

[0036] Table 1. Focal Length and Viewing Angle Comparison Table

[0037] focal length 1 / 3 CCD viewing angle 1 / 4 CCD viewing angle 2.8mm 89.9° 75.6° 4mm 69.9° 57° 6mm 50° 39.8° 8mm 38.5° 30.4° 12mm 26.2° 20.5°

[0038] (2) Based on the identification results of the type of each target detection object, the result of whether each target detection object is a continuous target is obtained.

[0039] Generally, signs, regardless of their specific type, are considered discontinuous targets. Security facilities, however, can be either continuous or discontinuous targets. For example, speed bumps in security facilities are discontinuous targets, while crash barriers are continuous targets. The distinction between continuous and discontinuous targets in this embodiment of the invention is as follows: since road images are collected every preset distance traveled, if the length of the detected target exceeds the preset distance, it is considered a continuous target; conversely, if the length of the detected target is less than the preset distance, it is considered a discontinuous target. The length of the detected target should be understood as its length along the direction of road extension.

[0040] (3) Based on whether each target detection object is a continuous target, the transformable index in the first sequence of each target detection object is obtained based on the number of shifts.

[0041] Specifically, this step includes the following two scenarios:

[0042] ① If the target object is a non-continuous target, then add the shift number to the index of the first element with a value of 0 after the consecutive elements with a value of 1 in the first sequence of the target object, to obtain a transformable index.

[0043] Let Index00 represent the index of the first element after a consecutive set of 1s that is 0. Then the transformable index is Index00 + P. T .

[0044] For example, if the first sequence of the target object is 011110000000, the index of the first element with a value of 0 after the consecutive elements with a value of 1 is 5, and the calculated shift number is 2, then the transformable index is 5+2=7.

[0045] ② If the target object is a continuous target, then add the shift number to the index of the consecutive elements with a value of 1 in the first sequence of the target object to obtain a transformable index.

[0046] Index10 to Index1n Let represent the index of an element that takes the value 1 consecutively. Then, the transformable index is Index10+P. T ~Index1 n +P T .

[0047] For example, if the first sequence of the target object is 011110000000, the indices of the elements with consecutive values ​​of 1 are 1 to 4, and the calculated shift number is 2, then the transformable indices are 1+2 to 4+2, that is, 3 to 6.

[0048] (4) Update all elements in the first sequence of each target detection object that were originally continuously set to 1 to 0, and update the elements corresponding to the transformable indices in the first sequence of each target detection object to 1, so as to obtain the second sequence of each target detection object.

[0049] The representation of the second sequence is consistent with that of the target sequence, and will not be elaborated further here.

[0050] For example, if the target to be detected is a non-continuous target, and the first sequence of the target to be detected is 011110000000, and the calculated shift is 2, then the second sequence is 000000010000.

[0051] For example, if the target to be detected is a continuous target, and the first sequence of the target to be detected is 011110000000, and the calculated shift is 2, then the second sequence is 000111100000.

[0052] Step S105: For each target detection object, compare the second sequence of the target detection object with the standard sequence of the target detection object in the database to determine whether the target detection object is missing.

[0053] Specifically, if the second sequence of the target detectable is exactly the same as the standard sequence of the target detectable, then the target detectable is determined to be not missing; otherwise, the target detectable is determined to be missing.

[0054] Preferably, the method of this embodiment of the invention further includes the following steps:

[0055] (1) Based on the identification results of the type of each target detection object, the result of whether each target detection object is a continuous target is obtained.

[0056] This step is the same as the previous one, so it will not be repeated here.

[0057] (2) Based on whether each target detection object is a continuous target, obtain the target index range from the second sequence of each target detection object.

[0058] Specifically, depending on whether it is a continuous objective, there are two cases:

[0059] ① If the target object is a non-continuous target, the target index range is the first target index range. The lower limit of the first target index range is the difference between the index of the element with a value of 1 in the second sequence of the target object and the preset parameter. The upper limit of the first target index range is the sum of the index of the element with a value of 1 in the second sequence of the target object and the preset parameter.

[0060] The preset parameters can be determined according to the relevant national or local standards for the target detection object. For example, the parameters for highway safety facilities should be determined according to the relevant regulations for highway safety facilities, and the parameters for warning signs should be determined according to the relevant regulations for warning signs.

[0061] Index00+P T Let represent the index of the element with a value of 1 in the second sequence. Then the range of the first target index is [Index00+P]. T -z,Index00+P T +z], where z represents a preset parameter, which is a positive number.

[0062] ② If the target object is a continuous target, the target index range is the second target index range. The lower limit of the second target index range is the index of the first element in the second sequence of the target object that has a continuous value of 1, and the upper limit of the second target index range is the index of the last element in the second sequence of the target object that has a continuous value of 1.

[0063] That is, the second target index range is [Index10+P] T Index1 n +P T ].

[0064] Preferably, embodiments of the present invention can also be based on Figure 3 The backtracking algorithm model shown performs a cross-check to determine whether signage and security facilities are missing. Based on this principle, such as... Figure 2 As shown, after step S105, the method of this embodiment of the invention further includes:

[0065] Step S106: If the target detection object is not missing, then obtain the second sequence of other target detection objects of the second type that are compatible with the first type of the target detection object.

[0066] Of the three types of signs, generally, only the area near warning signs will be equipped with security facilities, while in other cases, warning signs will be near security facilities.

[0067] Therefore, if the first type is a warning sign in the signage, then the second type is a security facility; or if the first type is a security facility, then the second type is a warning sign in the signage. That is, only when the target detection object is one of these two types will the second sequence of the target detection object of the corresponding type be obtained for mutual inspection.

[0068] Step S107: Compare the target index range of the first type of target detection object with the target index range of other target detection objects of the second type to obtain the result of whether the area where the first type of target detection object is located is missing other target detection objects of the second type.

[0069] Specifically, if the target index range of the first type of target detection object intersects with the target index range of other target detection objects of the second type, then it is determined that the area where the first type of target detection object is located contains other target detection objects of the second type; if the target index range of the first type of target detection object does not intersect with the target index range of other target detection objects of the second type, then it is determined that the area where the first type of target detection object is located lacks other target detection objects of the second type.

[0070] Because warning signs are non-continuous targets, security facilities include both continuous and non-continuous targets, denoted by Index0. 0,1 +P T This indicates the index of the warning sign in the second sequence, using [Index10+P]. T Index1 n +P T ] represents the index of a continuous security facility in the second sequence, using Index0. 0,2 +P T The index of the non-continuous security facility in the second sequence is represented by z1, and the preset parameters corresponding to the warning sign are represented by z2. This step can specifically have the following situations:

[0071] (1) The target object to be detected is a warning sign, and its target index range is [Index0]. 0,1 +P T -z1,Index0 0,1 +P T +z1], the second type of target detection object is a security facility, and it is a continuous target, so its target index range is [Index10+P]. T Index1 n +P T If [Index10+P] is an integer, then [Index10+P] is an integer. T Index1 n +P T Does it belong to or partially belong to [Index0]?0,1 +P T -z1,Index0 0,1 +P T If +z1], then the area where the warning sign is located has security facilities, meaning the security facilities are not missing; otherwise, the security facilities are missing.

[0072] (2) The target object to be detected is a warning sign, and its target index range is [Index0]. 0,1 +P T1 -z1,Index0 0,1 +P T1 +z1], the second type of target detection object is a security facility, and it is a non-continuous target, so its target index range is [Index0]. 0,2 +P T -z2,Index0 0,2 +P T +z2], then determine [Index0] 0,2 +P T -z2,Index0 0,2 +P T Does +z2 belong to or partially belong to [Index0]? 0,1 +P T1 -z1,Index0 0,1 +P T1 If +z1], then the area where the warning sign is located has security facilities, meaning the security facilities are not missing; otherwise, the security facilities are missing.

[0073] (3) If the target object is a security facility and is a non-continuous target, then its target index range is [Index0]. 0,2 +P T -z2,Index0 0,2 +P T +z2], the second type of target detection object is a warning sign, and its target index range is [Index0]. 0,1 +P T -z1,Index0 0,1 +P T +z1], then determine [Index0] 0,1 +P T -z1,Index0 0,1 +P T Does +z1 belong to or partially belong to [Index0]? 0,2 +P T -z2,Index0 0,2 +P T If the warning sign is present in the area where the security facility is located, then the warning sign is not missing; otherwise, the warning sign is missing.

[0074] (4) If the target object is a security facility and is a continuous target, then its target index range is [Index10+P]. T Index1 n +P T The second type of target detection object is a warning sign, and its target index range is [Index0]. 0,1 +P T -z1,Index0 0,1 +P T +z1], then determine [Index0] 0,1 +P T -z1,Index0 0,1 +P T Does +z1 belong to or partially belong to [Index10+P]? T Index1 n +P T If yes, then the area where the security facility is located has warning signs, meaning the warning signs are not missing; otherwise, the warning signs are missing.

[0075] In practical applications, in conjunction with relevant national standards, a backtracking decision card can be constructed by combining each target detection object and its corresponding target retrieval range, as well as the target detection objects and their corresponding target retrieval ranges that cooperate with it. The backtracking decision card includes: the target detection object and its location, and the target detection objects and their locations that cooperate with it, as shown in Table 2.

[0076] Table 2 Retrospective Decision Card

[0077]

[0078] For example, in application, the sign is a warning sign, and at the same time, the warning sign is a continuous sharp bend warning. In this case, the retrospective decision card mechanism can be activated to find the crash barrier that is matched with it from the retrospective decision card, and to determine whether the crash barrier matched with the warning sign is normal.

[0079] In addition, based on the detection results, the retrospective decision card can also include the status information of the target detection object and the target detection object that cooperates with it, i.e., whether it is missing or not missing.

[0080] This invention also discloses a computer-readable storage medium storing computer program instructions; when executed by a processor, the computer program instructions implement the method for detecting missing road signs and security facilities as described in the above embodiments.

[0081] This invention also discloses a detection system for missing road signs and security facilities, comprising: a computer-readable storage medium as described in the above embodiments.

[0082] In summary, the embodiments of the present invention can realize the automatic detection of missing signs and security facilities, and can perform mutual inspection of signs and security facilities, reducing manual intervention and improving detection and identification efficiency.

[0083] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting the absence of road signs and security installations, characterized in that, include: A database of road target detection objects is established, wherein the types of target detection objects include: signs and security facilities, and the database includes: a standard sequence of the geographical location of each target detection object and the category of each target detection object; During the operation of the inspection vehicle, an image of the road ahead of the inspection vehicle is captured by a camera installed on the vehicle every preset distance. In all the acquired images, each of the target detection objects is identified, and the location identification result and type identification result of each target detection object are recorded to obtain a first sequence of each target detection object composed of the location identification results; Based on the category identification results, the first sequence of each target detection object is modified to obtain the second sequence of each target detection object; For each target detection object, the second sequence of the target detection object is compared with the standard sequence of the target detection object in the database to determine whether the target detection object is missing. In this context, each element in the standard sequence, the first sequence, and the second sequence takes the value of 0 or 1. Each element corresponds to a geographical location, and each element is arranged in order of geographical location. If the value of an element is 0, it indicates that the geographical location does not have the target detection object. If the value of an element is 1, it indicates that the geographical location has the target detection object. The step of obtaining a second sequence for each of the target detection objects includes: For each of the target detection objects, a first sequence of translation positions is calculated, wherein the first sequence of translation positions is calculated by denotes the translation position, , c denotes the distance from the lane center line to the slope or the side line, θ denotes the visual angle of the camera, H denotes the image acquisition frequency, which is the preset distance / time. Based on the identification result of the type of each target detection object, the result of whether each target detection object is a continuous target is obtained; Based on whether each of the target detection objects is a continuous target, a transformable index in the first sequence of each target detection object is obtained based on the number of shifts; In the first sequence of each target detection object, all elements that were originally continuously 1 are updated to 0, and the elements corresponding to the transformable indices in the first sequence of each target detection object are updated to 1, thus obtaining the second sequence of each target detection object. The index represents the sequence number of the element.

2. The method of claim 1, wherein the method further comprises: determining a location of the road sign based on the image data; and determining a location of the security facility based on the image data. The step of obtaining a transformable index in the first sequence of each of the target detection objects includes: If the target object is a non-continuous target, then the index of the first element with a value of 0 after the consecutive elements with a value of 1 in the first sequence of the target object is shifted by a number of bits to obtain the transformable index.

3. The method for detecting missing road signs and safety facilities according to claim 1, characterized in that, The step of obtaining a transformable index in the first sequence of each of the target detection objects includes: If the target to be detected is a continuous target, then the indexes of the consecutive elements with a value of 1 in the first sequence of the target to be detected are respectively shifted by a number of bits to obtain the transformable index.

4. The method for detecting missing road signs and safety facilities according to claim 1, characterized in that, The step of determining whether the target detection object is missing includes: If the second sequence of the target detectable is exactly the same as the standard sequence of the target detectable, then the target detectable is determined to be not missing; otherwise, the target detectable is determined to be missing.

5. The method for detecting missing road signs and safety facilities according to claim 1, characterized in that, Also includes: Based on the identification result of the type of each target detection object, the result of whether each target detection object is a continuous target is obtained; Based on whether each of the target detection objects is a continuous target, the target index range is obtained from the second sequence of each of the target detection objects; Wherein, if the target object is a non-continuous target, the target index range is a first target index range, the lower limit of the first target index range is the difference between the index of the element with a value of 1 in the second sequence of the target object and a preset parameter, and the upper limit of the first target index range is the sum of the index of the element with a value of 1 in the second sequence of the target object and the preset parameter; Wherein, if the target object to be detected is a continuous target, then the target index range is the second target index range, the lower limit of the second target index range is the index of the first element in the second sequence of the target object that has a continuous value of 1, and the upper limit of the second target index range is the index of the last element in the second sequence of the target object that has a continuous value of 1; The index represents the sequence number of the element.

6. The method for detecting missing road signs and safety facilities according to claim 5, characterized in that, After the step of determining whether the target detector is missing, the method further includes: If the target detection object is not missing, then obtain a second sequence of other target detection objects of the second type that match the first type of the target detection object, wherein the first type is a warning sign in the signage and the second type is a security facility, or the first type is a security facility and the second type is a warning sign in the signage. The target index range of the first type of target detection object is compared with the target index range of other target detection objects of the second type to obtain the result of whether the area where the first type of target detection object is located is missing other target detection objects of the second type.

7. The method for detecting missing road signs and safety facilities according to claim 6, characterized in that, The step of obtaining the result of whether the area where the first type of target detection object is located is missing other target detection objects of the second type includes: If the target index range of the first type of target detection object intersects with the target index range of other target detection objects of the second type, then it is determined that the area where the first type of target detection object is located contains other target detection objects of the second type. If the target index range of the first type of target detection object does not intersect with the target index range of other target detection objects of the second type, then it is determined that the area where the first type of target detection object is located is missing other target detection objects of the second type.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, they implement the detection method for missing road signs and security facilities as described in any one of claims 1 to 7.

9. A detection system for missing road signs and security facilities, characterized in that, include: The computer-readable storage medium as described in claim 8.

Images