Road sign detection method and device

Abstract

The embodiment of the invention provides a road sign detection method and device, and the method comprises the steps: starting to collect a road sign image comprising a target road sign when the distance between the road sign and the target road sign is a preset first distance in a process of advancing towards the target road sign; after the vehicle travels and passes through the target road mark, when the vehicle is away from the target road mark by a preset second distance, stopping collecting the road mark image; for each frame of the collected road sign image, detecting a target frame corresponding to the target road sign in the image by using a pre-constructed target detection model; according to a target frame in each road sign image, determining a first position of a target road sign in each road sign image under the global coordinate system; and based on the first position of the target road sign in each road sign image, determining the estimated position of the target road sign in the target road section, and the method and the device can improve the detection accuracy of the road sign.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of highway facility maintenance, and in particular to a highway sign detection method and device. Background Art

[0002] With the acceleration of global urbanization and the continuous expansion of transportation networks, the construction and maintenance of highways and urban roads are particularly important. Road defects (such as cracks, potholes, subsidence, bulges, etc.) are one of the key factors affecting road quality and traffic safety. Timely and accurate identification and repair of these defects can not only effectively extend the service life of the road, but also reduce the incidence of traffic accidents and reduce maintenance costs.

[0003] At present, the road condition rapid detection system is generally used to identify and evaluate various road surface defects. During the data collection process, the equipment mainly uses positioning units and wheel speedometers to determine the location of highway signs, and marks specific road sections according to the location of highway signs, and further detects road defects on the road sections. Due to the limited positioning accuracy of the positioning unit and wheel speedometer, the position of the road section and the highway sign often deviates, making it difficult to accurately determine the specific location of the defect. Summary of the invention

[0004] In view of this, an object of the embodiments of the present application is to provide a highway sign detection method and device to solve the problem of deviation between the road section and the highway sign position.

[0005] Based on the above purpose, the embodiment of the present application provides a road sign detection method, including:

[0006] In the process of moving toward a target road sign, when the distance from the target road sign is a predetermined first distance, starting to collect road sign images including the target road sign; after moving past the target road sign, when the distance from the target road sign is a predetermined second distance, stopping collecting road sign images;

[0007] For each frame of highway sign image collected, a pre-built target detection model is used to detect a target frame corresponding to the target highway sign in the image;

[0008] Determine, according to the target frame in each highway sign image, a first position of the target highway sign in each highway sign image in the global coordinate system;

[0009] Based on the first position of the target highway sign in each highway sign image, an estimated position of the target highway sign in the target road segment is determined.

[0010] Optionally, determining the first position of the target road sign in each road sign image in the global coordinate system according to the target frame in each road sign image includes:

[0011] Establishing the global coordinate system with the position where the highway sign image is collected as the origin;

[0012] Convert the coordinates of the target frame in each highway sign image in the image coordinate system into the coordinates in the acquisition device coordinate system;

[0013] The coordinates of each target frame in the acquisition device coordinate system are converted into coordinates in the global coordinate system to obtain the first position of the target highway sign in each highway sign image.

[0014] Optionally, the highway sign image includes the target highway sign and other highway signs;

[0015] The method of detecting a target frame corresponding to the target highway sign in each frame of the highway sign image collected by the pre-built target detection model includes:

[0016] Detecting a target frame corresponding to the target highway sign and other frames corresponding to the other highway signs in the image using the target detection model;

[0017] The step of determining the first position of the target highway sign in each highway sign image in the global coordinate system according to the target frame in each highway sign image includes:

[0018] Convert the coordinates of each target frame and each other frame in the acquisition device coordinate system into coordinates in the global coordinate system;

[0019] In the global coordinate system, the coordinates within the predetermined area are used as the first positions of each target frame.

[0020] Optionally, after detecting the target frame corresponding to the target highway sign in the image by using the pre-built target detection model, the method further includes:

[0021] If there is a highway sign image in which the target frame is not detected, determining the position area of ​​the target frame in the highway sign image according to the target frames in the highway sign images of the previous frame and the next frame;

[0022] The target is detected again based on the position area to determine the position of the target frame in the highway sign image of the frame.

[0023] Optionally, after determining the first position of the target highway sign in each highway sign image in the global coordinate system, the method further includes:

[0024] The first position is processed by mean filtering to obtain a second position of the target highway sign in each highway sign image.

[0025] Optionally, determining the estimated position of the target highway sign in the target road section based on the first position of the target highway sign in each highway sign image includes:

[0026] The second position of the target highway sign in each highway sign image is subjected to RTS smoothing filtering processing to obtain an estimated position of the target highway sign in the target road section.

[0027] Optionally, after determining the estimated position of the target highway sign in the target road section, the method further includes:

[0028] A corresponding relationship between the target highway sign and the target road section is established according to the timestamp of collecting the highway sign image and the time of passing the target road section.

[0029] Optionally, the first distance and the second distance are determined according to a positioning result of a positioning unit and / or a detection result of a wheel speed meter.

[0030] Optionally, after detecting the target frame corresponding to the target highway sign in the image by using the pre-built target detection model, the method further includes:

[0031] According to the identification content of the target highway identification corresponding to the target frame, the highway identification images that do not belong to the target road section are filtered.

[0032] The present application also provides a road sign detection device, including:

[0033] A collection module, configured to start collecting a road sign image including the target road sign when the vehicle is at a predetermined first distance from the target road sign during the process of traveling toward the target road sign; and stop collecting the road sign image when the vehicle is at a predetermined second distance from the target road sign after passing the target road sign;

[0034] The target detection module is used to detect the target frame corresponding to the target highway sign in each frame of the highway sign image collected by the pre-built target detection model;

[0035] A first position processing module, used to determine a first position of a target highway sign in each highway sign image in a global coordinate system according to a target frame in each highway sign image;

[0036] The second position processing module is used to determine the estimated position of the target highway sign in the target road section based on the first position of the target highway sign in each highway sign image.

[0037] From the above description, it can be seen that the highway sign detection method and device provided in the embodiment of the present application, in the process of moving toward the target highway sign, when the distance from the target highway sign is a predetermined first distance, starts to collect the highway sign image including the target highway sign; after moving past the target highway sign, when the distance from the target highway sign is a predetermined second distance, stops collecting the highway sign image; for each frame of highway sign image collected, uses the target detection model to detect the target frame corresponding to the target highway sign in the image; according to the target frame in each highway sign image, determines the first position of the target highway sign in each highway sign image in the global coordinate system; based on the first position of the target highway sign in each highway sign image, determines the estimated position of the target highway sign in the target road section. The present application can improve the detection accuracy of highway signs, establish an accurate relationship between road sections and highway signs, and thus accurately determine the location of road defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 A schematic diagram of a method flow of an embodiment of the present application;

[0040] Figure 2 A simplified schematic diagram of the location of a highway sign according to an embodiment of the present application;

[0041] Figure 3 A schematic diagram of the device structure of an embodiment of the present application;

[0042] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0044] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connecting" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to represent relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0045] In the related art, a road condition rapid detection system mounted on a detection vehicle is used to identify and evaluate various road defects. During the driving process, the detection vehicle generally uses a GPS positioning unit and a wheel speedometer to determine the location of the highway sign. The system then determines the relationship between the road section and the highway sign based on the location of the highway sign and the road section being driven. When a defect is detected on the road, the specific location of the defect is determined based on the relationship between the road section and the highway sign. However, due to the positioning accuracy of the positioning unit and the wheel speedometer, the location of the located highway sign often deviates from its actual location, and the established relationship between the road section and the highway sign is not accurate, and thus the specific location of the defect cannot be accurately determined.

[0046] In view of this, an embodiment of the present application provides a highway sign detection method, which collects multiple frames of highway sign images in the process of passing a target highway sign, performs target detection based on the multiple frames of highway sign images, obtains a target frame of the target highway sign, determines the first position of the target highway sign in each highway sign image in the global coordinate system based on the target frame in each highway sign image, estimates the position of the target highway sign in the target road section based on the first position, and can accurately detect the position of the highway sign.

[0047] The technical solution of the present application is further described in detail below through specific embodiments.

[0048] like Figure 1 As shown, the road sign detection method provided in the embodiment of the present application includes:

[0049] S101: in the process of moving toward a target road sign, when the distance from the target road sign is a predetermined first distance, starting to collect road sign images including the target road sign; after moving past the target road sign, when the distance from the target road sign is a predetermined second distance, stopping collecting road sign images;

[0050] In this embodiment, the position of the highway sign is detected by an electronic device mounted on a collection device (e.g., a detection vehicle, a drone, a patrol robot, etc.), that is, the execution subject of the method of this application is an electronic device. Highway signs are set at predetermined positions on the highway, for example, a kilometer post is set every 1 kilometer on the highway, and the name and kilometer of the highway are marked with the kilometer post, which plays a role in highway positioning.

[0051] like Figure 2 As shown, taking the detection vehicle as an example, when the target highway sign is detected by the electronic device, the detection vehicle 100 travels toward the target highway sign 10 along the extension direction of the road. When the distance from the target highway sign 10 reaches a first distance, the detection function of the electronic device is turned on, and the highway sign image including the target highway sign is collected by the electronic device, and the collected highway sign image is processed to detect the position of the highway sign; when the detection vehicle 100 passes the target highway sign 10 and the distance of the detection vehicle 100 away from the target highway sign 10 reaches a second distance, the detection function of the electronic device is stopped, including stopping the collection of the highway sign image and image processing related functions. According to the above process, by dynamically turning on and off the detection function, each highway sign is detected in turn, which can not only save resource overhead, but also avoid a large number of false detections at locations without highway signs, and avoid the problem of card points existing in manual collection. For example, when the highway is marked with kilometer posts, since there is no need to detect between kilometer posts, the detection function can be periodically turned on and off according to the deployment location of the kilometer posts. This not only enables the detection of all kilometer posts, but also avoids a large number of false detections due to unnecessary detection between kilometer posts, which brings new problems to detection and saves unnecessary expenses.

[0052] In some embodiments, the electronic device includes an image acquisition unit for acquiring images of highway signs, a target detection module for detecting highway signs, a positioning unit for locating the distance of the highway signs, and a wheel speed meter, etc. During the driving process of the detection vehicle, the first distance and the second distance between the detection vehicle and the target highway sign are determined according to the positioning result of the positioning unit and / or the detection result of the wheel speed meter.

[0053] In some embodiments, the first distance and the second distance can be determined by weighted summation based on the positioning result of the GPS positioning unit and the positioning result of the wheel speed meter. Specifically, when the number of satellites is greater than or equal to 4, the positioning result of the GPS positioning unit is more accurate, and the weight of the positioning result of the wheel speed meter can be set to 0, that is, the first distance and the second distance are directly determined based on the positioning result of the GPS positioning unit; when the number of satellites is greater than 0 and less than 4, the satellite positioning accuracy is not so high, and the weight of the positioning result of the GPS positioning unit can be set to 0.6, and the weight of the positioning result of the wheel speed meter can be set to 0.4, and the positioning results of the GPS positioning unit and the wheel speed meter are weighted summed to determine the first distance and the second distance; when the number of satellites is less than 0 (for example, the inspection vehicle enters a tunnel), the weight of the GPS positioning unit can be set to 0, and the position of the inspection vehicle can be determined directly based on the positioning result of the wheel speed meter, thereby determining the first distance and the second distance.

[0054] In some embodiments, the collection device is equipped with a rapid road condition detection system and an electronic device used to implement highway sign detection in the present application. Alternatively, the rapid road condition detection system is implemented based on the electronic device, and the rapid road condition detection system can implement the highway sign detection method of the present application. The specific implementation method is not limited.

[0055] S102: for each frame of highway sign image collected, using a pre-built target detection model to detect a target frame corresponding to the target highway sign in the image;

[0056] In this embodiment, during the process of the electronic device being turned on and off, multiple frames of road sign images are continuously collected, and the position of the road sign is detected based on the multiple frames of road sign images. For each frame of the road sign image, the target detection model is used to detect the target frame corresponding to the target road sign in the image, that is, each frame of the road sign image is input into the target detection model, and the target detection model performs target detection on the image to identify the target frame corresponding to the target road sign in the image.

[0057] In some methods, the YOLO model can be used to take highway signs as the target, and a manually labeled highway sign data set can be used for training. After training, a target detection model for identifying a target frame with a highway sign as the target is obtained. At the same time, in order to identify the sign content on the highway sign, based on the detected target frame, an OCR text recognition model can be used to identify the sign content of the highway sign in the target frame, and then the recognition result of the highway sign is obtained. In other methods, a multimodal large model can be used to take highway signs as the target, and a data set of highway signs and their sign content can be used for training. After training, a target detection model for identifying a target frame with a highway sign as the target and the sign content of the highway sign can be obtained, that is, the detection result of the target detection model includes the target frame of the highway sign and the sign content in the target frame. The above is only an exemplary description, and the specific network model selected for the target detection model is not limited, and the structure and training method of the network model are not specifically described.

[0058] In some embodiments, the detection vehicle equipped with electronic equipment is traveling on the target road section. When the detection vehicle is 100 meters away from the target road sign, the image acquisition unit and the target detection model are started, the road sign image is acquired by the image acquisition unit, the target detection model is used to detect the target frame corresponding to the target road sign in the road sign image, and the mark content in the target frame is identified. After the detection vehicle passes the target road sign and is 100 meters away from the target road sign, the image acquisition unit and the target detection model are turned off, and the detection of the target road sign is stopped. The detection vehicle continues to travel, and detects each road sign in turn according to the above process until the detection task of the road signs on all target sections is completed.

[0059] S103: determining a first position of a target highway sign in each highway sign image in a global coordinate system according to a target frame in each highway sign image;

[0060] In this embodiment, after collecting multiple frames of highway sign images and performing target detection on each highway sign image to obtain a target frame corresponding to the target highway sign in the image, the first position of the target highway sign in each highway sign image in the global coordinate system is determined by converting the coordinates. Specifically, determining the first position of the target highway sign in each highway sign image in the global coordinate system includes:

[0061] Establish a global coordinate system with the position where the highway sign image collection starts as the origin;

[0062] Convert the coordinates of the target frame in each highway sign image in the image coordinate system into the coordinates in the acquisition device coordinate system;

[0063] The coordinates of each target frame in the acquisition device coordinate system are converted into coordinates in the global coordinate system to obtain the first position of the target highway sign in each highway sign image.

[0064] In this embodiment, when the acquisition device moves to a first distance from the target highway sign, it starts to acquire the highway sign image, takes the current position of the acquisition device as the origin, establishes a global coordinate system, detects the highway sign image, obtains the target frame and its coordinates in the image coordinate system, converts the coordinates of the target frame in the image coordinate system to the coordinates in the acquisition device coordinate system according to the relationship between the image coordinate system and the acquisition device coordinate system, and converts the coordinates of the target frame in the acquisition device coordinate system to the coordinates in the global coordinate system according to the relationship between the acquisition device coordinate system and the global coordinate system, and obtains the first position of the target highway sign in the highway sign image in the global coordinate system. Among them, the method for establishing the global coordinate system, the coordinate transformation between the image coordinate system, the acquisition device coordinate system and the global coordinate system belong to the common technologies in the field of visual processing, and this application does not provide a specific explanation.

[0065] In some embodiments, the highway sign image includes a target highway sign and other highway signs;

[0066] For each frame of highway sign image collected, the pre-built target detection model is used to detect the target frame corresponding to the target highway sign in the image, including:

[0067] Using the target detection model to detect the target box corresponding to the target highway sign and other boxes corresponding to other highway signs in the image;

[0068] Determining a first position of a target highway sign in each highway sign image in a global coordinate system according to a target frame in each highway sign image includes:

[0069] Convert the coordinates of each target frame and each other frame in the acquisition device coordinate system into the coordinates in the global coordinate system;

[0070] In the global coordinate system, the coordinates within the predetermined area are used as the first position of each target frame.

[0071] like Figure 2As shown, in this embodiment, during the movement of the acquisition device, the electronic device acquires multiple frames of highway sign images. Since there are multiple highway signs on the highway, the acquired highway sign images may include both the target highway sign 10 to be acquired and other highway signs 20. After the highway sign images are input into the target detection model, the detection results of the model include both the target frame corresponding to the identified target highway sign and other frames corresponding to other highway signs. The coordinates of the target frame and other frames in the image coordinate system are converted into coordinates in the acquisition device coordinate system, and then the coordinates of the target frame and other frames in the acquisition device coordinate system are converted into coordinates in the global coordinate system. In the global coordinate system, the position of the highway sign is stationary. If the existence of errors is not considered, the position of the same highway sign in the global coordinate system should be the same, that is, the position of the target highway sign detected in each highway sign image in the global coordinate system should be the same, for example, both are located at At point A, the positions of other road signs detected in each road sign image in the global coordinate system should also be the same, for example, all located at point B. Taking into account the existence of errors, the target road signs detected in each road sign image are actually distributed within a certain area range in the global coordinate system, for example, located within a certain range near point A. Based on this, after obtaining the coordinates of the target frame and other frames in the global coordinate system, the target frame and other frames are clustered according to the range of coordinates, and the coordinates within the predetermined area are used as the coordinates of the target frame of the corresponding target road sign detected in each road sign image, and then other road signs in the image are excluded and no subsequent processing is performed.

[0072] In some embodiments, after detecting the target frame corresponding to the target highway sign in the image using the pre-built target detection model, the method further includes:

[0073] If there is a highway sign image in which the target frame is not detected, determining the position area of ​​the target frame in the highway sign image according to the target frames in the highway sign images of the previous frame and the next frame;

[0074] The target is detected again based on the position area to determine the position of the target frame in the highway sign image of the frame.

[0075] In this embodiment, considering that there may be multiple interferences in the image, the target detection model may fail to detect the target frame corresponding to the target highway sign, that is, there may be a missed detection. Since the position of the target highway sign in the global coordinate system is static and unchanged, for a missed highway sign image, the position area where the target frame is located in the missed highway sign image can be estimated based on the target frame of the previous highway sign image and the target frame of the next highway sign image, and then the target detection is re-performed based on the position area to determine the position of the missed target frame, thereby achieving frame filling in the missed highway sign image.

[0076] In some embodiments, re-detecting the target based on the position area to determine the position of the target frame in the frame of the highway sign image includes: in the frame of the highway sign image, expanding the area to be detected based on the position area according to a certain ratio, re-detecting the target based on the area to be detected, and determining the position of the target frame in the area to be detected. If the target frame is still not detected in the area to be detected, then supplementing the target frame at a predetermined position in the position area of ​​the frame of the highway sign image, for example, determining the predetermined position of the supplementary target frame according to the average of the positions of the target frames in the previous frame and the next frame of the highway sign image.

[0077] In some embodiments, target detection is performed on each frame of highway sign images collected, and after the image of the missed target frame is determined, each frame of the missed image is individually framed. After all the missed images are framed, the coordinates of the target frames are converted to a global coordinate system based on all highway sign images with target frames. In the global coordinate system, the coordinates within a predetermined area are used as the first position of each target frame, and the estimated position of the target highway sign is determined based on the first position. Frame filling can improve the accuracy of highway sign detection.

[0078] S104: Determine an estimated position of the target highway sign in the target road section based on the first position of the target highway sign in each highway sign image.

[0079] In this embodiment, after the first position of the target highway sign in each highway sign image is determined, the estimated position of the target highway sign in the target road section is determined based on the first position of the target highway sign in each image.

[0080] In some embodiments, after determining the first position of the target road sign in each road sign image in the global coordinate system, the method further includes: performing mean filtering on the first position to obtain the second position of the target road sign in each road sign image. That is, performing mean filtering on the first position of the target road sign in each road sign image in the global coordinate system to obtain the second position of the target road sign in each road sign image, thereby improving the accuracy of position detection.

[0081] In some embodiments, determining an estimated position of the target highway sign in the target road segment based on the first position of the target highway sign in each highway sign image includes:

[0082] The second position of the target highway sign in each highway sign image is subjected to RTS smoothing filtering processing to obtain an estimated position of the target highway sign in the target road section.

[0083] In this embodiment, RTS smoothing filtering is performed based on the second position of the target highway sign in each highway sign image, that is, Kalman filtering is first performed on the second position of each frame of highway sign image to obtain the filtered position, and then reverse filtering is performed from the position of the last frame image to the position of the first frame image to complete the RTS smoothing filtering process and obtain the best estimated position of the target highway sign. Through global smoothing filtering, the accuracy of highway sign position estimation can be greatly improved.

[0084] In some embodiments, after determining the estimated position of the target highway sign in the target road segment, the method further includes:

[0085] According to the timestamp of the highway sign image acquisition and the time of passing the target road section, a corresponding relationship between the target highway sign and the target road section is established.

[0086] In this embodiment, after determining the estimated position of the target highway sign, according to the timestamp of collecting the highway sign image and the time of the target road section passed by the collecting device, for example, the detection vehicle passes through the target road section 1 in the time period T, and collects multiple frames of highway sign images at time points t1-tn in the time period T, and determines the estimated position of the target highway sign according to the multiple frames of highway sign images, then the association relationship between the target highway sign and the target road section can be established, and the target road section can be located using the target highway sign. Since the position of the detected highway sign is relatively accurate, an accurate correspondence relationship between the road section and the highway sign can be established, and then the specific location of the road disease can be accurately determined. Optionally, according to the timestamp of collecting the highway sign image and the time of passing the target road section, a highway sign number label can be added to the position corresponding to the target highway sign in the highway sign image to establish a correspondence relationship between the target highway sign and the target road section.

[0087] In some embodiments, after detecting the target frame corresponding to the target highway sign in the image using the target detection model, the method further includes:

[0088] According to the sign content of the target highway sign corresponding to the target frame, the highway sign images that do not belong to the target road section are filtered.

[0089] In this embodiment, taking into account that the acquisition device may pass through different road sections, when detecting highway signs in the target road section, after detecting the target frame corresponding to the target highway sign in the image through the target detection model, the acquired image is filtered according to the identification content corresponding to the target highway sign in the target frame, and images that do not belong to the target road section are removed. Subsequent detection and processing are not performed on the images that do not belong to the target road section, thereby improving the accuracy of highway sign detection and improving data processing efficiency.

[0090] The highway sign detection method provided in the embodiment of the present application detects the target highway signs passed by by dynamically turning on and off the electronic device, which can save computing resources and reduce the false detection rate; target detection is performed on the collected highway sign image to obtain the target frame corresponding to the target highway sign in the image, and the missed image is framed, and the position of each target frame in the global coordinate system is determined based on all the filled target frames, and the accuracy of the position is improved by using mean filtering, and the position is optimized using the RTS smoothing filter algorithm, which can greatly improve the accuracy of the position of the highway sign.

[0091] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0092] It should be noted that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0093] like Figure 3 As shown, the embodiment of the present application also provides a road sign detection device, including:

[0094] The acquisition module is used to start acquiring a road sign image including the target road sign when the vehicle is at a predetermined first distance from the target road sign during the process of moving toward the target road sign; and stop acquiring the road sign image when the vehicle is at a predetermined second distance from the target road sign after passing the target road sign;

[0095] The target detection module is used to detect the target frame corresponding to the target highway sign in the image using a pre-built target detection model for each highway sign image collected;

[0096] A first position processing module, used to determine a first position of a target highway sign in each highway sign image in a global coordinate system according to a target frame in each highway sign image;

[0097] The second position processing module is used to determine the estimated position of the target highway sign in the target road section based on the first position of the target highway sign in each highway sign image.

[0098] For the convenience of description, the above devices are described in terms of functions divided into various modules. Of course, when implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0099] The apparatus of the above-mentioned embodiment is used to implement the corresponding method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0100] Figure 4 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0101] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0102] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0103] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0104] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0105] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0106] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0107] The electronic device of the above embodiment is used to implement the corresponding method in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0108] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0109] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0110] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present disclosure, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0111] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0112] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present disclosure.

Claims

1. A road sign detection method, characterized in that: include: In the process of moving toward a target road sign, when the distance from the target road sign is a predetermined first distance, starting to collect road sign images including the target road sign; after moving past the target road sign, when the distance from the target road sign is a predetermined second distance, stopping collecting road sign images; For each frame of highway sign image collected, a pre-built target detection model is used to detect a target frame corresponding to the target highway sign in the image; Determine, according to the target frame in each highway sign image, a first position of the target highway sign in each highway sign image in the global coordinate system; Based on the first position of the target highway sign in each highway sign image, an estimated position of the target highway sign in the target road segment is determined.

2. The method according to claim 1, characterized in that Determining a first position of a target highway sign in each highway sign image in a global coordinate system according to a target frame in each highway sign image includes: Establishing the global coordinate system with the position where the highway sign image is collected as the origin; Convert the coordinates of the target frame in each highway sign image in the image coordinate system into the coordinates in the acquisition device coordinate system; The coordinates of each target frame in the acquisition device coordinate system are converted into coordinates in the global coordinate system to obtain the first position of the target highway sign in each highway sign image.

3. The method according to claim 2, characterized in that The highway sign image includes the target highway sign and other highway signs; The method of detecting a target frame corresponding to the target highway sign in each frame of the highway sign image collected by the pre-built target detection model includes: Detecting a target frame corresponding to the target highway sign and other frames corresponding to the other highway signs in the image using the target detection model; The step of determining the first position of the target highway sign in each highway sign image in the global coordinate system according to the target frame in each highway sign image includes: Convert the coordinates of each target frame and each other frame in the acquisition device coordinate system into coordinates in the global coordinate system; In the global coordinate system, the coordinates within the predetermined area are used as the first positions of each target frame.

4. The method according to claim 1, characterized in that After detecting the target frame corresponding to the target highway sign in the image by using the pre-built target detection model, the method further includes: If there is a highway sign image in which the target frame is not detected, determining the position area of ​​the target frame in the highway sign image according to the target frames in the highway sign images of the previous frame and the next frame; The target is detected again based on the position area to determine the position of the target frame in the highway sign image of the frame.

5. The method according to claim 1, characterized in that After determining the first position of the target highway sign in each highway sign image in the global coordinate system, the method further includes: The first position is processed by mean filtering to obtain a second position of the target highway sign in each highway sign image.

6. The method according to claim 5, characterized in that The step of determining an estimated position of the target highway sign in the target road section based on the first position of the target highway sign in each highway sign image includes: The second position of the target highway sign in each highway sign image is subjected to RTS smoothing filtering processing to obtain an estimated position of the target highway sign in the target road section.

7. The method according to claim 1, characterized in that After determining the estimated position of the target highway sign in the target road section, the method further includes: A corresponding relationship between the target highway sign and the target road section is established according to the timestamp of collecting the highway sign image and the time of passing the target road section.

8. The method according to claim 1, characterized in that The first distance and the second distance are determined according to a positioning result of a positioning unit and / or a detection result of a wheel speed meter.

9. The method according to claim 1, characterized in that: After detecting the target frame corresponding to the target highway sign in the image by using the pre-built target detection model, the method further includes: According to the identification content of the target highway identification corresponding to the target frame, the highway identification images that do not belong to the target road section are filtered.

10. A road sign detection device, characterized in that: include: A collection module, configured to start collecting a road sign image including the target road sign when the vehicle is at a predetermined first distance from the target road sign during the process of traveling toward the target road sign; and stop collecting the road sign image when the vehicle is at a predetermined second distance from the target road sign after passing the target road sign; The target detection module is used to detect the target frame corresponding to the target highway sign in each frame of the highway sign image collected by the pre-built target detection model; A first position processing module, used to determine a first position of a target highway sign in each highway sign image in a global coordinate system according to a target frame in each highway sign image; The second position processing module is used to determine the estimated position of the target highway sign in the target road section based on the first position of the target highway sign in each highway sign image.

Images