A road foreign matter detection method, system, electronic device and storage medium
Patent Information
- Application Number
- CN202411799080.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-09
- Publication Date
- 2026-08-18
- Estimated Expiration
- 2044-12-09
AI Technical Summary
[0002]道路异物检测存在以下问题：1)由于异常物体属性的不确定性，想要尽可能的收集各种各样的异常物体，需要耗费大量的人力财力，不太现实
[0009]The method of this invention combines the long and short-term background of road images with prior information to detect foreign objects on the road, avoiding the problem of low accuracy caused by collecting a large number of sample images of abnormal objects on the road. It improves the accuracy of road foreign object detection while adapting to different complex environmental scenarios.
Smart Images
Figure CN119784997B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of deep learning technology, and in particular to a method, system, electronic device, and storage medium for detecting foreign objects on roads. Background Technology
[0002] The following problems exist in road foreign object detection: 1) Due to the uncertainty of the attributes of anomalous objects, collecting as many different types of anomalous objects as possible would require a large amount of manpower and resources, which is impractical. 2) Road environments are outdoor environments, complex and changeable. The scene changes with the time of day and is affected by seasons and weather (cloudy, sunny, foggy, etc.). 3) Traditional detection methods require a reliable and realistic background. Compared to indoor applications, extracting a robust background in this complex and changeable outdoor road scene is very challenging. When the background is not updated or the update rate is slow, changes in lighting and shadows on the road surface can lead to false detections. Moreover, if the background contains targets, "ghosting" may occur. When the background is updated frequently, static anomalous objects may be added to the background, leading to missed detections. 4) Deep learning-based anomalous object detection (such as YOLO series, RCNN series, etc.) can only output anomalous objects of fixed categories and are powerless against anomalous objects outside their category range.
[0003] Therefore, there is an urgent need to provide a technical solution to address the above problems. Summary of the Invention
[0004] To address the aforementioned technical problems, this invention provides a method, system, electronic device, and storage medium for detecting foreign objects on roads.
[0005] Firstly, the present invention provides a method for detecting foreign objects on roads, the technical solution of which is as follows:
[0006] Acquire the image to be detected containing the target road area, and extract the long-term background image and short-term background image corresponding to the image to be detected;
[0007] Based on the image to be detected, the long-term background image, and the short-term background image, and using a trained road object detection model, the road object detection result of the image to be detected is obtained; wherein, the road object detection model includes: a convolutional neural network for extracting prior information of the image and a target detection network for road object detection.
[0008] The beneficial effects of the road foreign object detection method of the present invention are as follows:
[0009] The method of this invention combines the long and short-term background of road images with prior information to detect foreign objects on the road, avoiding the problem of low accuracy caused by collecting a large number of sample images of abnormal objects on the road. It improves the accuracy of road foreign object detection while adapting to different complex environmental scenarios.
[0010] The long and short-term background information and prior information corresponding to road images can improve the accuracy of foreign object detection on roads.
[0011] Based on the above solution, the road foreign object detection method of the present invention can be further improved as follows.
[0012] In one optional approach, the step of obtaining the long-term background image and the short-term background image corresponding to the image to be detected includes:
[0013] The long-term background image corresponding to the image to be detected is obtained using a first Gaussian mixture model, and the short-term background image corresponding to the image to be detected is obtained using a second Gaussian mixture model; wherein, both the first Gaussian mixture model and the second Gaussian mixture model are trained using sample images containing the target road region, and the update frequency of the first Gaussian mixture model is less than the update frequency of the second Gaussian mixture model.
[0014] In one alternative embodiment, the convolutional neural network includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a first group convolutional layer, a second group convolutional layer, and a third group convolutional layer; the object detection network includes: a first backbone layer, a second backbone layer, a third backbone layer, a first channel connection layer, a second channel connection layer, a third channel connection layer, a first neck layer, a second neck layer, a first head layer, a second head layer, and a third head layer;
[0015] The first convolutional layer is used to: perform convolution processing on the target input image to obtain a first image feature; wherein, the target input image is a nine-channel image composed of the image to be detected, the long-term background image, and the short-term background image; the first grouped convolutional layer is used to: perform grouped convolution processing on the first image feature to obtain a second image feature; the second convolutional layer is used to: perform convolution processing on the second image feature to obtain a third image feature; the third convolutional layer is used to: perform convolution processing on the first image feature to obtain a fourth image feature; the second grouped convolutional layer is used to: perform grouped convolution processing on the fourth image feature to obtain a fifth image feature; the fourth convolutional layer is used to: perform convolution processing on the fifth image feature to obtain a sixth image feature; the fifth convolutional layer is used to: perform convolution processing on the fourth image feature to obtain a seventh image feature; the third grouped convolutional layer is used to: perform grouped convolution processing on the seventh image feature to obtain an eighth image feature; the sixth convolutional layer is used to: perform convolution processing on the eighth image feature to obtain a ninth image feature.
[0016] The first Backbone layer is used to: extract features from the target input image to obtain the tenth image feature; the first channel connection layer is used to: perform channel connection on the third image feature and the tenth image feature to obtain the eleventh image feature; the second Backbone layer is used to: extract features from the eleventh image feature to obtain the twelfth image feature; the second channel connection layer is used to: perform channel connection on the sixth image feature and the twelfth image feature to obtain the thirteenth image feature; the third Backbone layer is used to: extract features from the thirteenth image feature to obtain the fourteenth image feature; the third channel connection layer is used to: perform channel connection on the ninth image feature and the fourteenth image feature to obtain the fifteenth image feature. Image features; the first Neck layer is used to: fuse the eleventh, twelfth, and fourteenth image features to obtain the sixteenth image feature; the first Head layer is used to: detect the sixteenth image feature to obtain a first detection result; the second Neck layer is used to: fuse the thirteenth and fourteenth image features to obtain the seventeenth image feature; the second Head layer is used to: detect the seventeenth image feature to obtain a second detection result; the third Head layer is used to: detect the fifteenth image feature to obtain a third detection result; wherein, the first detection result, the second detection result, and the third detection result together constitute the road foreign object detection result.
[0017] In one alternative approach, the road foreign object detection result is: the target detection box containing the foreign object in the image to be detected and the probability of the target detection box.
[0018] Secondly, the present invention provides a road foreign object detection system, the technical solution of which is as follows:
[0019] Includes: an acquisition module and a detection module;
[0020] The acquisition module is used to: acquire a detection image containing the target road area, and extract the long-term background image and short-term background image corresponding to the detection image;
[0021] The detection module is used to: obtain the road object detection result of the image to be detected based on the image to be detected, the long-term background image, and the short-term background image, and using a trained road object detection model; wherein, the road object detection model includes: a convolutional neural network for extracting prior information of the image and a target detection network for road object detection.
[0022] The beneficial effects of the road foreign object detection system of the present invention are as follows:
[0023] The system of this invention combines the long and short-term background of road images with prior information to detect foreign objects on the road, avoiding the problem of low accuracy caused by collecting a large number of sample images of abnormal objects on the road. While adapting to different complex environmental scenarios, it improves the accuracy of road foreign object detection.
[0024] Based on the above solution, the road foreign object detection system of the present invention can be further improved as follows.
[0025] In one alternative approach, the acquisition module is specifically used for:
[0026] The long-term background image corresponding to the image to be detected is obtained using a first Gaussian mixture model, and the short-term background image corresponding to the image to be detected is obtained using a second Gaussian mixture model; wherein, both the first Gaussian mixture model and the second Gaussian mixture model are trained using sample images containing the target road region, and the update frequency of the first Gaussian mixture model is less than the update frequency of the second Gaussian mixture model.
[0027] In one alternative embodiment, the convolutional neural network includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a first group convolutional layer, a second group convolutional layer, and a third group convolutional layer; the object detection network includes: a first backbone layer, a second backbone layer, a third backbone layer, a first channel connection layer, a second channel connection layer, a third channel connection layer, a first neck layer, a second neck layer, a first head layer, a second head layer, and a third head layer;
[0028] The first convolutional layer is used to: perform convolution processing on the target input image to obtain a first image feature; wherein, the target input image is a nine-channel image composed of the image to be detected, the long-term background image, and the short-term background image; the first grouped convolutional layer is used to: perform grouped convolution processing on the first image feature to obtain a second image feature; the second convolutional layer is used to: perform convolution processing on the second image feature to obtain a third image feature; the third convolutional layer is used to: perform convolution processing on the first image feature to obtain a fourth image feature; the second grouped convolutional layer is used to: perform grouped convolution processing on the fourth image feature to obtain a fifth image feature; the fourth convolutional layer is used to: perform convolution processing on the fifth image feature to obtain a sixth image feature; the fifth convolutional layer is used to: perform convolution processing on the fourth image feature to obtain a seventh image feature; the third grouped convolutional layer is used to: perform grouped convolution processing on the seventh image feature to obtain an eighth image feature; the sixth convolutional layer is used to: perform convolution processing on the eighth image feature to obtain a ninth image feature.
[0029] The first Backbone layer is used to: extract features from the target input image to obtain the tenth image feature; the first channel connection layer is used to: perform channel connection on the third image feature and the tenth image feature to obtain the eleventh image feature; the second Backbone layer is used to: extract features from the eleventh image feature to obtain the twelfth image feature; the second channel connection layer is used to: perform channel connection on the sixth image feature and the twelfth image feature to obtain the thirteenth image feature; the third Backbone layer is used to: extract features from the thirteenth image feature to obtain the fourteenth image feature; the third channel connection layer is used to: perform channel connection on the ninth image feature and the fourteenth image feature to obtain the fifteenth image feature. Image features; the first Neck layer is used to: fuse the eleventh, twelfth, and fourteenth image features to obtain the sixteenth image feature; the first Head layer is used to: detect the sixteenth image feature to obtain a first detection result; the second Neck layer is used to: fuse the thirteenth and fourteenth image features to obtain the seventeenth image feature; the second Head layer is used to: detect the seventeenth image feature to obtain a second detection result; the third Head layer is used to: detect the fifteenth image feature to obtain a third detection result; wherein, the first detection result, the second detection result, and the third detection result together constitute the road foreign object detection result.
[0030] In one alternative approach, the road foreign object detection result is: the target detection box containing the foreign object in the image to be detected and the probability of the target detection box.
[0031] Thirdly, the technical solution of an electronic device according to the present invention is as follows:
[0032] It includes a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of the road foreign object detection method of the present invention.
[0033] Fourthly, the technical solution of a computer-readable storage medium provided by the present invention is as follows:
[0034] The computer-readable storage medium stores instructions that, when read, cause the computer-readable storage medium to perform the steps of the road foreign object detection method of the present invention.
[0035] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description
[0036] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0037] Figure 1 This is a schematic flowchart of an embodiment of a road foreign object detection method according to the present invention;
[0038] Figure 2 A schematic diagram of the overall process of the road foreign object detection model;
[0039] Figure 3 This is a schematic diagram of an embodiment of a road foreign object detection system according to the present invention;
[0040] Figure 4 This is a schematic diagram of an embodiment of an electronic device according to the present invention. Detailed Implementation
[0041] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.
[0042] Figure 1 This diagram illustrates a flowchart of an embodiment of a road foreign object detection method provided by the present invention. This method can be executed by an electronic device such as a terminal device or a server. The terminal device can be any fixed or mobile terminal, such as a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, vehicle-mounted device, or wearable device. The server can be a single server or a server cluster consisting of multiple servers. Any electronic device can implement the road foreign object detection method by having its processor call computer-readable instructions stored in its memory. Figure 1 As shown, it includes the following steps:
[0043] S1. Obtain the image to be detected containing the target road area, and extract the long-term background image and short-term background image corresponding to the image to be detected.
[0044] The target road area is the road area to be detected for foreign objects in this embodiment. Foreign objects refer to objects other than common road objects such as vehicles, pedestrians, and traffic signs. A long-term background image is a collection of pixels in an image whose grayscale values remain almost unchanged or only slightly altered. It typically represents the real external environment and remains stable over a relatively long period. A short-term background image is an image in which the background portion is captured or generated within a short time; it may contain more dynamic changes and noise. Long-term background images prioritize stability and accuracy, while short-term background images prioritize real-time performance and dynamism.
[0045] S2. Based on the image to be detected, the long-term background image, and the short-term background image, and using the trained road object detection model, obtain the road object detection result of the image to be detected.
[0046] The road foreign object detection model includes a convolutional neural network for extracting prior information from images and a target detection network for detecting foreign objects on the road.
[0047] In one alternative approach, the steps of obtaining the long-term background image and the short-term background image corresponding to the image to be detected include:
[0048] The first Gaussian mixture model is used to obtain the long-term background image corresponding to the image to be detected, and the second Gaussian mixture model is used to obtain the short-term background image corresponding to the image to be detected.
[0049] Both the first Gaussian mixture model and the second Gaussian mixture model are trained using sample images containing the target road area. The update frequency of the first Gaussian mixture model is lower than that of the second Gaussian mixture model.
[0050] In one alternative approach, such as Figure 2 As shown, the convolutional neural network includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a first group convolutional layer, a second group convolutional layer, and a third group convolutional layer; the object detection network includes: a first backbone layer, a second backbone layer, a third backbone layer, a first channel connection layer, a second channel connection layer, a third channel connection layer, a first neck layer, a second neck layer, a first head layer, a second head layer, and a third head layer.
[0051] The first convolutional layer is used to: perform convolution processing on the target input image to obtain the first image feature; wherein, the target input image is a nine-channel image composed of the image to be detected, the long-term background image, and the short-term background image; the first grouped convolutional layer is used to: perform grouped convolution processing on the first image feature to obtain the second image feature; the second convolutional layer is used to: perform convolution processing on the second image feature to obtain the third image feature; the third convolutional layer is used to: perform convolution processing on the first image feature to obtain the fourth image feature; the second grouped convolutional layer is used to: perform grouped convolution processing on the fourth image feature to obtain the fifth image feature; the fourth convolutional layer is used to: perform convolution processing on the fifth image feature to obtain the sixth image feature; the fifth convolutional layer is used to: perform convolution processing on the fourth image feature to obtain the seventh image feature; the third grouped convolutional layer is used to: perform grouped convolution processing on the seventh image feature to obtain the eighth image feature; the sixth convolutional layer is used to: perform convolution processing on the eighth image feature to obtain the ninth image feature.
[0052] The first backbone layer is used to extract features from the target input image to obtain the tenth image feature; the first channel connection layer is used to perform channel concatenation of the third and tenth image features to obtain the eleventh image feature; the second backbone layer is used to extract features from the eleventh image feature to obtain the twelfth image feature; the second channel connection layer is used to perform channel concatenation of the sixth and twelfth image features to obtain the thirteenth image feature; the third backbone layer is used to extract features from the thirteenth image feature to obtain the fourteenth image feature; the third channel connection layer is used to extract features from the ninth image feature. The 11th, 12th, and 14th image features are concatenated to obtain the 15th image feature. The first Neck layer is used to fuse the 11th, 12th, and 14th image features to obtain the 16th image feature. The first Head layer is used to detect the 16th image feature to obtain the first detection result. The second Neck layer is used to fuse the 13th and 14th image features to obtain the 17th image feature. The second Head layer is used to detect the 17th image feature to obtain the second detection result. The third Head layer is used to detect the 15th image feature to obtain the third detection result.
[0053] The first, second, and third test results together constitute the road foreign object detection results.
[0054] In one alternative approach, the road foreign object detection result is: the target detection box containing the foreign object in the image to be detected and the probability of the target detection box.
[0055] When there is a foreign object in the image to be detected, there is at least one target detection box. Each target detection box corresponds to a category and a probability. Each category of the target detection box corresponds to a type of foreign object. The probability of the target detection box indicates the probability that the target detection box belongs to a certain category. The higher the probability value, the more realistic the detection result is.
[0056] The technical solution of this embodiment combines the long and short-term background of road images with prior information to detect foreign objects on the road. This avoids the problem of low accuracy caused by collecting a large number of sample images of abnormal objects on the road. While adapting to different complex environmental scenarios, it improves the accuracy of road foreign object detection.
[0057] Figure 3 A schematic diagram of an embodiment of a road foreign object detection system 200 provided by the present invention is shown. Figure 3 As shown, the system 200 includes: an acquisition module 210 and a detection module 220;
[0058] The acquisition module 210 is used to: acquire the image to be detected containing the target road area, and extract the long-term background image and short-term background image corresponding to the image to be detected;
[0059] The detection module 220 is used to: obtain the road object detection result of the image to be detected based on the image to be detected, the long-term background image and the short-term background image, and using a trained road object detection model; wherein, the road object detection model includes: a convolutional neural network for extracting prior information of the image and a target detection network for road object detection.
[0060] In one alternative approach, module 210 is specifically used for:
[0061] The first Gaussian mixture model is used to obtain the long-term background image corresponding to the image to be detected, and the second Gaussian mixture model is used to obtain the short-term background image corresponding to the image to be detected. Both the first and second Gaussian mixture models are trained using sample images containing the target road area, and the update frequency of the first Gaussian mixture model is lower than that of the second Gaussian mixture model.
[0062] In one alternative embodiment, the convolutional neural network includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a first group convolutional layer, a second group convolutional layer, and a third group convolutional layer; the object detection network includes: a first backbone layer, a second backbone layer, a third backbone layer, a first channel connection layer, a second channel connection layer, a third channel connection layer, a first neck layer, a second neck layer, a first head layer, a second head layer, and a third head layer;
[0063] The first convolutional layer is used to: perform convolution processing on the target input image to obtain the first image feature; wherein, the target input image is a nine-channel image composed of the image to be detected, the long-term background image, and the short-term background image; the first grouped convolutional layer is used to: perform grouped convolution processing on the first image feature to obtain the second image feature; the second convolutional layer is used to: perform convolution processing on the second image feature to obtain the third image feature; the third convolutional layer is used to: perform convolution processing on the first image feature to obtain the fourth image feature; the second grouped convolutional layer is used to: perform grouped convolution processing on the fourth image feature to obtain the fifth image feature; the fourth convolutional layer is used to: perform convolution processing on the fifth image feature to obtain the sixth image feature; the fifth convolutional layer is used to: perform convolution processing on the fourth image feature to obtain the seventh image feature; the third grouped convolutional layer is used to: perform grouped convolution processing on the seventh image feature to obtain the eighth image feature; the sixth convolutional layer is used to: perform convolution processing on the eighth image feature to obtain the ninth image feature;
[0064] The first backbone layer is used to extract features from the target input image to obtain the tenth image feature; the first channel connection layer is used to perform channel concatenation of the third and tenth image features to obtain the eleventh image feature; the second backbone layer is used to extract features from the eleventh image feature to obtain the twelfth image feature; the second channel connection layer is used to perform channel concatenation of the sixth and twelfth image features to obtain the thirteenth image feature; the third backbone layer is used to extract features from the thirteenth image feature to obtain the fourteenth image feature; the third channel connection layer is used to perform channel concatenation of the ninth and fourteenth image features to obtain the eleventh image feature. The system comprises 15 image features; the first Neck layer is used to fuse the 11th, 12th, and 14th image features to obtain the 16th image feature; the first Head layer is used to detect the 16th image feature to obtain the first detection result; the second Neck layer is used to fuse the 13th and 14th image features to obtain the 17th image feature; the second Head layer is used to detect the 17th image feature to obtain the second detection result; and the third Head layer is used to detect the 15th image feature to obtain the third detection result. The first, second, and third detection results together constitute the road foreign object detection result.
[0065] In one alternative approach, the road foreign object detection result is: the target detection box containing the foreign object in the image to be detected and the probability of the target detection box.
[0066] It should be noted that the beneficial effects of the road foreign object detection system provided in the above embodiments are the same as those of the road foreign object detection method described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.
[0067] The road foreign object detection system of the present invention can be a computer program (including program code) running on a computer device. For example, the road foreign object detection system of the present invention is an application software that can be used to execute the corresponding steps in the road foreign object detection method of the present invention.
[0068] In some embodiments, the road foreign object detection system of the present invention can be implemented in a combination of hardware and software. As an example, the road foreign object detection system of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the road foreign object detection method of the present invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.
[0069] The modules described in the embodiments of this invention can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.
[0070] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the road foreign object detection methods described above. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the road foreign object detection method shown in any embodiment of the present invention by calling the computer program.
[0071] In one alternative embodiment, an electronic device is provided, such as Figure 4 As shown, Figure 4 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.
[0072] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0073] Bus 4002 may include a path for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus 4002 is represented by only one thick line, but this does not mean that there is only one bus or one type of bus.
[0074] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.
[0075] The memory 4003 stores the application code (computer program) for executing the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.
[0076] Among them, electronic devices can also be terminal devices. A terminal device can be any terminal device that can install applications and access web pages through applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.
[0077] It should be noted that, Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.
[0078] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the road foreign object detection methods described above.
[0079] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.
[0080] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the aforementioned road foreign object detection method.
[0081] Computer program code for performing the operations of this invention can be written in one or more programming languages or a combination thereof. These programming languages include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0082] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0083] The computer-readable storage medium provided in this invention can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0084] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.
[0085] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.
[0086] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.
[0087] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.
[0088] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A method for detecting foreign objects on roads, characterized in that, include: Acquire the image to be detected containing the target road area, and extract the long-term background image and short-term background image corresponding to the image to be detected; Based on the image to be detected, the long-term background image, and the short-term background image, and using a trained road object detection model, the road object detection result of the image to be detected is obtained; wherein, the road object detection model includes: a convolutional neural network for extracting prior information of the image and a target detection network for road object detection; The convolutional neural network includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a first group convolutional layer, a second group convolutional layer, and a third group convolutional layer; the object detection network includes: a first backbone layer, a second backbone layer, a third backbone layer, a first channel connection layer, a second channel connection layer, a third channel connection layer, a first neck layer, a second neck layer, a first head layer, a second head layer, and a third head layer; The first convolutional layer is used to: perform convolution processing on the target input image to obtain a first image feature; wherein, the target input image is a nine-channel image composed of the image to be detected, the long-term background image, and the short-term background image; the first grouped convolutional layer is used to: perform grouped convolution processing on the first image feature to obtain a second image feature; the second convolutional layer is used to: perform convolution processing on the second image feature to obtain a third image feature; the third convolutional layer is used to: perform convolution processing on the first image feature to obtain a fourth image feature; the second grouped convolutional layer is used to: perform grouped convolution processing on the fourth image feature to obtain a fifth image feature; the fourth convolutional layer is used to: perform convolution processing on the fifth image feature to obtain a sixth image feature; the fifth convolutional layer is used to: perform convolution processing on the fourth image feature to obtain a seventh image feature; the third grouped convolutional layer is used to: perform grouped convolution processing on the seventh image feature to obtain an eighth image feature; the sixth convolutional layer is used to: perform convolution processing on the eighth image feature to obtain a ninth image feature. The first Backbone layer is used to: extract features from the target input image to obtain the tenth image feature; the first channel connection layer is used to: perform channel connection on the third image feature and the tenth image feature to obtain the eleventh image feature; the second Backbone layer is used to: extract features from the eleventh image feature to obtain the twelfth image feature; the second channel connection layer is used to: perform channel connection on the sixth image feature and the twelfth image feature to obtain the thirteenth image feature; the third Backbone layer is used to: extract features from the thirteenth image feature to obtain the fourteenth image feature; the third channel connection layer is used to: perform channel connection on the ninth image feature and the fourteenth image feature to obtain the fifteenth image feature. Image features; the first Neck layer is used to: fuse the eleventh, twelfth, and fourteenth image features to obtain the sixteenth image feature; the first Head layer is used to: detect the sixteenth image feature to obtain a first detection result; the second Neck layer is used to: fuse the thirteenth and fourteenth image features to obtain the seventeenth image feature; the second Head layer is used to: detect the seventeenth image feature to obtain a second detection result; the third Head layer is used to: detect the fifteenth image feature to obtain a third detection result; wherein, the first detection result, the second detection result, and the third detection result together constitute the road foreign object detection result.
2. The road foreign object detection method according to claim 1, characterized in that, The steps of obtaining the long-term background image and short-term background image corresponding to the image to be detected include: The long-term background image corresponding to the image to be detected is obtained using a first Gaussian mixture model, and the short-term background image corresponding to the image to be detected is obtained using a second Gaussian mixture model; wherein, both the first Gaussian mixture model and the second Gaussian mixture model are trained using sample images containing the target road region, and the update frequency of the first Gaussian mixture model is less than the update frequency of the second Gaussian mixture model.
3. The road foreign object detection method according to claim 1 or 2, characterized in that, The road foreign object detection result is: the target detection box containing the foreign object in the image to be detected and the probability of the target detection box.
4. A road foreign object detection system, characterized in that, include: Acquisition module and detection module; The acquisition module is used to: acquire a detection image containing the target road area, and extract the long-term background image and short-term background image corresponding to the detection image; The detection module is used to: obtain the road object detection result of the image to be detected based on the image to be detected, the long-term background image, and the short-term background image, and using a trained road object detection model; wherein, the road object detection model includes: a convolutional neural network for extracting prior information of the image and a target detection network for road object detection; The convolutional neural network includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a first group convolutional layer, a second group convolutional layer, and a third group convolutional layer; the object detection network includes: a first backbone layer, a second backbone layer, a third backbone layer, a first channel connection layer, a second channel connection layer, a third channel connection layer, a first neck layer, a second neck layer, a first head layer, a second head layer, and a third head layer; The first convolutional layer is used to: perform convolution processing on the target input image to obtain a first image feature; wherein, the target input image is a nine-channel image composed of the image to be detected, the long-term background image, and the short-term background image; the first grouped convolutional layer is used to: perform grouped convolution processing on the first image feature to obtain a second image feature; the second convolutional layer is used to: perform convolution processing on the second image feature to obtain a third image feature; the third convolutional layer is used to: perform convolution processing on the first image feature to obtain a fourth image feature; the second grouped convolutional layer is used to: perform grouped convolution processing on the fourth image feature to obtain a fifth image feature; the fourth convolutional layer is used to: perform convolution processing on the fifth image feature to obtain a sixth image feature; the fifth convolutional layer is used to: perform convolution processing on the fourth image feature to obtain a seventh image feature; the third grouped convolutional layer is used to: perform grouped convolution processing on the seventh image feature to obtain an eighth image feature; the sixth convolutional layer is used to: perform convolution processing on the eighth image feature to obtain a ninth image feature. The first Backbone layer is used to: extract features from the target input image to obtain the tenth image feature; the first channel connection layer is used to: perform channel connection on the third image feature and the tenth image feature to obtain the eleventh image feature; the second Backbone layer is used to: extract features from the eleventh image feature to obtain the twelfth image feature; the second channel connection layer is used to: perform channel connection on the sixth image feature and the twelfth image feature to obtain the thirteenth image feature; the third Backbone layer is used to: extract features from the thirteenth image feature to obtain the fourteenth image feature; the third channel connection layer is used to: perform channel connection on the ninth image feature and the fourteenth image feature to obtain the fifteenth image feature. Image features; the first Neck layer is used to: fuse the eleventh, twelfth, and fourteenth image features to obtain the sixteenth image feature; the first Head layer is used to: detect the sixteenth image feature to obtain a first detection result; the second Neck layer is used to: fuse the thirteenth and fourteenth image features to obtain the seventeenth image feature; the second Head layer is used to: detect the seventeenth image feature to obtain a second detection result; the third Head layer is used to: detect the fifteenth image feature to obtain a third detection result; wherein, the first detection result, the second detection result, and the third detection result together constitute the road foreign object detection result.
5. The road foreign object detection system according to claim 4, characterized in that, The acquisition module is specifically used for: The long-term background image corresponding to the image to be detected is obtained using a first Gaussian mixture model, and the short-term background image corresponding to the image to be detected is obtained using a second Gaussian mixture model; wherein, both the first Gaussian mixture model and the second Gaussian mixture model are trained using sample images containing the target road region, and the update frequency of the first Gaussian mixture model is less than the update frequency of the second Gaussian mixture model.
6. The road foreign object detection system according to claim 4 or 5, characterized in that, The road foreign object detection result is: the target detection box containing the foreign object in the image to be detected and the probability of the target detection box.
7. An electronic device, characterized in that, The electronic device includes a processor coupled to a memory storing at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the road foreign object detection method as described in any one of claims 1 to 3.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer-readable storage medium to implement the road foreign object detection method as described in any one of claims 1 to 3.
Citation Information
Patent Citations
Double-background model and improved YOLO-P remnant detection method
CN118864811A