Target detection method based on large-screen display of sensor

Abstract

A target detection method based on a novel self-developed ponding sensor is provided; the implementation method comprises the following steps of: firstly, installing a sensor circuit, introducing an LM393 voltage comparator, connecting a slide rheostat at an anti-phase end to adjust and judge whether a voltage threshold value of accumulated water exists or not, designing an accumulated water sensing device at an in-phase input end, and providing an alarm for subsequent detection work; an SN74LVC1G04DBVR phase inverter is additionally arranged at the output end of the circuit and is used for changing high and low levels output by the circuit; the method comprises the following steps of: designing a detection logic circuit, assembling according to a designed GPIO (General Purpose Input / Output) interface, finally designing a corresponding PCB (Printed Circuit Board) and 3D (Three-Dimensional) simulation, detecting a ponding target, and displaying the ponding target detection through a large screen.

Description

Technical Field

[0001] The invention belongs to the field of target detection and relates to a target detection method for a sensor large-screen display. Background Art

[0002] In recent years, with the progress of science and technology and the improvement of people's economic level, more and more people choose to drive to travel in order to achieve more convenient, faster and higher quality travel needs. However, there are certain safety hazards on the road, such as road cracks, rolling stones on the mountain, vehicle spills, pedestrians, animals, and obstacles of stationary vehicles, or sudden natural bad weather such as rain, snow, ice, and water accumulation. These will pose a great threat to people's lives, property and safety, and cause immeasurable losses to people's life, property safety and travel time efficiency. How to drive more safely and with lower risks has become a major problem that needs to be solved urgently. In order to solve the above-mentioned various road problems, the concept of Intelligent Traffic System (ITS) was born. Intelligent Traffic System is the product of the combination of various cutting-edge scientific and technological technologies, including information communication transmission technology, sensor technology, new energy technology and computer science. Intelligent Traffic System provides traffic management personnel and drivers with real-time road information by adopting advanced technology, while also enhancing the safety of road travel and ensuring traffic efficiency. And it also includes the target detection technology of sensors.

[0003] Traditional target detection systems rely on manually defined target features, which makes target detection face complex challenges in the early stages. Because the features are set manually, they cannot effectively express image features. When faced with complex and diverse data sets, feature setting becomes more cumbersome, which limits the generalization ability of traditional target detection algorithms. Even if a variety of optimization strategies are adopted, it is still difficult to reduce the computational burden of finding complex features. In addition, although the sliding window algorithm ensures a certain level of detection accuracy, a large number of sliding frame operations also increase the demand for computer computing power accordingly. Target detection using sensors can use target detection algorithms based on deep learning, which are responsible for collecting data from the environment as the core of the Internet of Things. It can collect important information such as temperature, humidity, position, speed, visual images, and sound in real time. The large amount of data collected by the sensor is transmitted through the Internet of Things network and then aggregated to the cloud platform or edge computing device for processing. Through data mining, machine learning and artificial intelligence algorithms, these data can be analyzed to facilitate the identification of patterns, development trends and anomalies, and the results of data analysis can be used to make predictions about the future or make decisions about current situations. The detection system corrects and adjusts itself based on the analysis results and the impact of the decision. The present invention innovates a self-developed sensor for auxiliary judgment, and at the same time adds a protocol logic chip to the self-developed water accumulation sensor, develops an expansion board dedicated to the GPIO interface, and combines it with the temperature and humidity sensor SHT3x-ARP of the Swiss Sensirion company to greatly improve the accuracy of road water accumulation and rain and snow detection. Summary of the invention

[0004] The purpose of the present invention is to provide a large-screen display target detection method based on a new self-developed sensor. To provide a new detection method for road conditions that cannot be detected by artificial intelligence, making road monitoring more convenient and quick, the present invention adopts the following technical solutions.

[0005] Step 1: Install the sensor circuit.

[0006] In the design of water accumulation sensor, the core is the design of comparator circuit. In this circuit design, a LM393 voltage comparator is first introduced, which can compare the voltage at the same phase terminal and the voltage at the opposite phase terminal, and output the comparison result in the form of high and low levels.

[0007] A sliding rheostat is connected to the inverting end to adjust the voltage threshold for determining whether there is water accumulation. A water accumulation sensing device is designed at the inverting input end. It consists of two electrodes DET_A and DET_B with opposite polarities placed at a certain distance. One electrode is grounded and the other is connected to the inverting input end. At the same time, an LED alarm device is connected to the output end of the comparator to judge the high and low levels of the output. The high level is on and the low level is off. Assuming the total voltage is 10V, when there is no water between the two electrodes, the resistance composed of the two electrodes DET_A and DET_B is infinite by default. According to the circuit voltage division principle, the voltage divided by the inverting end approaches 10V. At this time, the voltage at the inverting end is greater than the voltage at the reverse end, and the output end outputs a high level, and the LED light is on. When there is water between the two electrodes, it is equivalent to having a medium between the two electrodes, which is equivalent to the existence of resistance. The voltage threshold is set by adjusting the sliding rheostat at the inverting end so that the voltage at the inverting end is just greater than the voltage at the inverting end. At this time, the output end outputs a low level and the LED light is off.

[0008] Since the water accumulation sensor node circuit is designed to detect the absence of water when it is high level and the presence of water when it is low level, in order to facilitate subsequent work, an SN74LVC1G04DBVR inverter is added to the output end of the circuit to change the high and low levels of the circuit output.

[0009] In addition, in the design of the circuit, if there is water between the two plates, it will change its structure and change its on-off. However, the range of the two plates is too small, so the two plates are expanded to 18, DET_A and DET_B are divided into two groups of nine according to Figure 5 Place it in the right position.

[0010] Assume that the red electrode sheet is DET_A and the blue electrode sheet is DET_B. They are placed on the plates in a cross-spaced manner. The plates are placed at different positions for water accumulation detection, which increases the detection area. Such a circuit integrated with several modules is the designed water accumulation sensor node circuit. No matter where the water drops, as long as any two red and blue electrodes with different polarities can be connected, a comparator circuit can be formed, thereby achieving the purpose of detecting water accumulation.

[0011] After designing the water accumulation sensor node circuit, four identical sensor nodes are combined together, and an SN74LVC32APWR AND gate logic chip is introduced. The four sensor nodes are operated with an AND gate logic in pairs, and the output results are operated with an AND operation again. Finally, the signal of the entire water accumulation sensor detecting water accumulation is output.

[0012] It can be found that no matter which of the four water accumulation sensor nodes detects the water accumulation signal, it will eventually output the detected water accumulation signal through logical operations. In such a circuit, when the D0 interface of each sensor node has no output, the circuit will be in a floating state between high and low levels. The floating state of D0 is pulled to a low level position through a pull-down resistor to the ground, ensuring the accuracy of the circuit output level. At the same time, a capacitor device is designed between the power supply and the ground as a buffer to ensure the stability of the entire circuit operation.

[0013] Step 2: Assemble the sensor.

[0014] The GPIO interface is used to connect sensor peripheral devices to the core devices of the system. Peripheral devices that use I2C and self-developed protocols include two IoT sensors: water accumulation sensor and temperature and humidity sensor. IoT sensors do not require high-speed data connections, but require stable connections to ensure that the data transmission process is not disturbed. The GPIO interface contains a large number of protocols, and the interface definition of each protocol is different, so the corresponding interface of the protocol needs to be brought out. At the same time, it is necessary to add a protocol logic chip to the self-developed water accumulation sensor to ensure the normal operation of the sensor, so a dedicated expansion version was developed for this system.

[0015] The developed expansion board is connected to the GPIO interface of the core device through a 40PIN interface, and the I2C, SPI, and CAN interfaces are brought out. A protocol logic chip is set on the expansion board so that the self-developed protocol can be connected to the core device through the chip. Each expansion board can connect four sets of water accumulation sensors with self-developed protocols, and the other brought-out interfaces provide the possibility of adding other sensors. The GPIO expansion board and the core device are placed in the cabinet, and the other connected sensors are brought out from the cabinet by inserting the corresponding interfaces of the expansion board.

[0016] It can be found that each water accumulation detection node can be divided into two polarities, and each polarity can be connected to 9 water accumulation detection nodes. The water accumulation detection node can be activated by connecting water accumulation detection plates of different polarities. The activated water accumulation detection node transmits the signal to the expansion board, and the core device receives the auxiliary judgment signal of water accumulation. Each detection board can connect 4 water accumulation detection nodes, so a system can connect 36 groups of water accumulation detection plates, and any group of systems that detect water accumulation will receive an auxiliary judgment signal of water accumulation.

[0017] Step 3: Design the corresponding PCB and 3D simulation to detect water accumulation targets, and display the water accumulation target detection results on a large screen.

[0018] Altium Designer software was used to design the corresponding PCB layout. Considering the size and volume of the sensor should be as small as possible, the aesthetics of the sensor, and the scalability and flexibility requirements for future upgrades or modifications, the layout and routing of the components in the layout were carefully designed, and finally a PCB layout that can ensure the circuit board has strong performance and efficient operation was obtained.

[0019] The LED screen controller uses the Dahua DH-PHTOC-PT11 control card, which is compatible with the Nova traffic protocol and needs to be developed based on this communication protocol. The traffic protocol has a plug-and-play host computer program, but it is incompatible with the Nvidia Jetson AGX Orin architecture and system used in this system, and needs to be redeveloped. The control card comes with a complete communication protocol source code. After transplantation and testing, the control protocol is successfully connected to the core device used in this system. The information dissemination module is also responsible for the LED screen layout, cutting the overlong character string and displaying it in large fonts on the corresponding LED screen.

[0020] Step 4: Combine hardware and software solutions.

[0021] The nodes of this system include cameras, sensors, edge computing devices, antennas and interactive devices. The interactive device uses a large-area LED bulletin board with strong anti-reflection ability to ensure that the driver can see the interactive content clearly from a distance; the sensor uses self-developed or finished sensors to provide physical sensors and artificial intelligence methods for road infrastructure warning and road abnormality warning, and form redundancy to ensure the normal operation of the warning function; Camera: Use a camera with 1080P resolution and infrared night vision function to provide effective protection for night road detection and curve warning. This system has flexible configuration. Each node can freely combine multiple edge computing devices, cameras, sensors, and interactive devices according to different scales to achieve on-demand configuration. For example, in the curve warning scenario, two groups of cameras and two groups of edge computing devices can be assumed on both sides of the curve, and the sensors can be arranged according to actual conditions.

[0022] In order to achieve the modularity and scalability of the system, this system takes edge computing devices as the core, switches and expansion boards as node switching devices, LED screens, surveillance cameras, water accumulation sensors and temperature and humidity sensors as peripheral devices. All three devices have the ability to be replaced or updated individually to meet flexible deployment in complex usage environments.

[0023] The LED screen and surveillance camera are connected to the switch, and communicate with the edge computing device in two directions through the RJ-45 interface; the water accumulation sensor, temperature and humidity sensor, and IoT sensor peripherals are connected to the expansion board to provide the edge computing device with the ability to perceive the surrounding environment, enhance the accuracy of machine learning judgment, and reduce misjudgments and missed judgments. The switch is connected to each connected device through a network cable, which has the advantages of strong anti-interference ability and long-distance laying. The expansion board is connected to the edge computing device through a dedicated 40PIN cable and placed next to the edge computing device. The connection with the sensor uses the self-developed protocol and I2C protocol, both of which have a certain degree of anti-interference ability.

[0024] The software indicators are to detect cars, trucks, pedestrians, traffic cones, two-wheelers, three-wheelers, convertibles, night vehicles, dogs, cows, sheep, cracks, ice and snow road obstacles or road anomalies. In order to reasonably allocate the computing power of edge computing devices, the artificial intelligence part uses advanced target detection algorithms to reduce the amount of calculation while ensuring accuracy, so that the delay of output information is further reduced when the power consumption of edge computing devices is reduced. The software framework is divided into four parts: system detection module, Kafka module, system analysis module and system information dissemination module. Among them, the system detection module is responsible for artificial intelligence algorithm target detection, Kafka is the information transmission module, the system analysis module is the post-processing module of the system, including sensor information post-processing and information release post-processing, and the information dissemination module is mainly responsible for the docking of the system main program output and the LED screen.

[0025] 1. System detection module

[0026] The system detection module is an artificial intelligence processing module. It uses DeepStream as the framework and develops a pruning optimization algorithm based on the YOLOV8 network. It optimizes the accuracy and speed of the original network framework so that it can be deployed on the edge computing module for operation.

[0027] 2. Kafka module

[0028] To ensure the stability of the system, the program is divided into several subroutines for execution. The artificial intelligence part is executed by the system detection module, which outputs the detection results to the system analysis module through Kafka for analysis, and then outputs the analysis results. Adding subroutine modules can avoid the main program crash and thus the system crash, and can also make the system more stable.

[0029] A typical Kafka architecture includes several message producers, several node servers, several message consumers, and a ZooKeeper cluster. Kafka manages cluster configuration, elects leaders, and performs rebalance (i.e., consumer load balancing) when consumer groups change through ZooKeeper. Producers use the push mode to publish messages to brokers, and consumers use the pull mode to subscribe to and consume messages from brokers. In this system, Kafka's Producer is the system detection module, and Consumer is the system analysis module. The system detection module outputs the detection result string, which is passed to the system analysis module through Kafka. The protocol layer that communicates with Kafka has been embedded in the system detection module and can be directly connected to Kafka-Server. Before Kafka is started, the front-end process ZooKeeper needs to be started. After ZooKeeper is started, Kafka is pulled up and the message transmission function is started. According to system requirements, a daemon is written. If Kafka stops working, Kafka is restarted to ensure stable operation of the system.

[0030] 3. System analysis module

[0031] This module is a post-processing module, including sensor information post-processing and information release post-processing. Sensor information post-processing includes further weighted processing of sensor information and conditional judgment processing, and information release post-processing includes post-processing of message data transmitted by the system detection module.

[0032] The strings transmitted by the system detection module through Kafka mainly include the result information of the identified objects, as well as the identification information of ice, snow and water accumulation. These strings are all real-time identification information. Due to the restrictions of the screen refresh rate and the output information rules, directly outputting the real-time detection information will result in disorganized information, which will directly affect the curve warning scene and the driver will not be able to obtain effective information. Therefore, the information needs to be cached and processed, and the processed information can be handed over to the information dissemination module and finally displayed on the LED screen.

[0033] Sensor post-processing not only includes comparing and weighting the water accumulation, ice and snow warnings output by the artificial intelligence network with the original data, but also includes a watchdog program to judge whether the sensor transmits error frames or the sensor does not respond. If the number of transmitted error frames or empty frames is greater than 5 times, a reset command is issued to the sensor.

[0034] Information post-release processing is a comprehensive display strategy processing process for various information. If the result string transmitted by the system detection module is legal and weighted with the original sensor data, the result frequency and sensor data parameters appearing in the result are judged, and the result is processed according to the display strategy, and finally output to the information dissemination module in the form of a string and displayed on the LED screen.

[0035] 4. Information dissemination module

[0036] The information dissemination module is mainly responsible for the connection between the system main program output and the LED screen. The LED screen controller uses the Dahua DH-PHTOC-PT11 control card, which is compatible with the Nova traffic protocol and needs to be developed based on this communication protocol. The traffic protocol has a plug-and-play host computer program, but it is incompatible with the Nvidia Jetson AGX Orin architecture and system used in this system, and needs to be redeveloped. The control card comes with a complete communication protocol source code. After transplantation and testing, the control protocol is successfully connected to the core device used in this system. The information dissemination module is also responsible for the LED screen layout, cutting the extra-long character string and displaying it in large font on the corresponding LED screen to remind the drivers of passing vehicles to drive carefully and pass slowly.

[0037] The present invention first designs the water accumulation sensor node circuit, then performs related designs, designs the circuit of the water accumulation sensor detection electrode, and then designs the water accumulation detection circuit and related interfaces to facilitate the connection between the circuit and the physical object, and finally performs target detection and displays it on a large screen, and finally simulates to verify the effectiveness and accuracy of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 It is a flow chart of the present invention;

[0040] Figure 2 It is the voltage comparator of LM393;

[0041] Figure 3 This is the node circuit diagram of the water accumulation sensor;

[0042] Figure 4 Design diagram for reverse circuit;

[0043] Figure 5 It is the diagram for the plate placement;

[0044] Figure 6 This is the design diagram of the water accumulation detection circuit;

[0045] Figure 7 This is the logic circuit diagram for water accumulation detection;

[0046] Figure 8 is a diagram of a capacitor device;

[0047] Fig. 9 This is the connection diagram of the water sensor;

[0048] Fig.10 This is the simulation diagram of the sensor node circuit not detecting water accumulation;

[0049] Fig.11 Simulation diagram of the sensor node circuit detecting water accumulation. DETAILED DESCRIPTION

[0050] In order to make the above and other purposes, features and advantages of the present invention more obvious, embodiments of the present invention are specifically cited below and described in detail with reference to the accompanying drawings.

[0051] The purpose of the present invention is to provide a target detection method based on a new self-developed water accumulation sensor. To provide a new detection method for road conditions that cannot be detected by artificial intelligence, making road monitoring more convenient and quick, in order to achieve the above purpose, the present invention adopts the following technical solutions. A target detection method based on a new self-developed water accumulation sensor has the following steps:

[0052] Step 1: Install the sensor circuit.

[0053] In the design of water accumulation sensor, the core is the design of comparator circuit. In this circuit design, a LM393 voltage comparator is first introduced, which can compare the voltage at the same phase terminal and the voltage at the opposite phase terminal, and output the comparison result in the form of high and low levels. The effective circuit diagram is as follows Figure 2 shown.

[0054] A sliding rheostat is connected to the inverting end to adjust the voltage threshold for determining whether there is water accumulation. A water accumulation sensing device is designed at the inverting input end. It consists of two electrodes DET_A and DET_B with opposite polarities placed at a certain distance. One electrode is grounded and the other is connected to the inverting input end. At the same time, an LED alarm device is connected to the output end of the comparator to judge the high and low levels of the output. The high level is on and the low level is off. Assuming the total voltage is 10V, when there is no water between the two electrodes, the resistance composed of the two electrodes DET_A and DET_B is infinite by default. According to the circuit voltage division principle, the voltage divided by the inverting end approaches 10V. At this time, the voltage at the inverting end is greater than the voltage at the reverse end, and the output end outputs a high level, and the LED light is on. When there is water between the two electrodes, it is equivalent to having a medium between the two electrodes, which is equivalent to the existence of resistance. The voltage threshold is set by adjusting the sliding rheostat at the inverting end so that the voltage at the inverting end is just greater than the voltage at the inverting end. At this time, the output end outputs a low level and the LED light is off. The node circuit design of the water accumulation sensor is as follows: Figure 3 shown.

[0055] Since the water accumulation sensor node circuit is designed to detect the absence of water at a high level and the presence of water at a low level, in order to facilitate subsequent work, an SN74LVC1G04DBVR inverter is added to the output end of the circuit to change the high and low levels of the circuit output. Figure 4 shown.

[0056] In addition, in the design of the circuit, if there is water between the two plates, it will change its structure and change its on-off. However, the range of the two plates is too small, so we expand the two plates to 18, DET_A and DET_B are divided into two groups of nine according to Figure 5 Place it in the right position.

[0057] Assume that the red electrode piece is DET_A and the blue electrode piece is DET_B. They are placed on the plates in a cross-spaced manner. The plates are placed in different positions for water accumulation detection, which increases the detection area. Such a circuit integrated with several modules is the designed water accumulation sensor node circuit. No matter where the water drops, as long as any two red and blue electrodes with different polarities can be connected, a comparator circuit can be formed to achieve the purpose of detecting water accumulation. The circuit design diagram is shown below. Figure 6 shown.

[0058] After designing the water accumulation sensor node circuit, four identical sensor nodes are combined together, and an SN74LVC32APWR AND gate logic chip is introduced. The four sensor nodes are operated by an AND gate in pairs, and the output result is operated again. Finally, the signal of the entire water accumulation sensor detecting water accumulation is output. The water accumulation detection logic circuit is as follows: Figure 7 shown.

[0059] It can be seen that no matter which of the four water accumulation sensor nodes detects the water accumulation signal, it will eventually output the detected water accumulation signal through logical operations. In such a circuit, when the DO interface of each sensor node has no output, the circuit will be in a floating state between high and low levels. The floating state of DO is pulled to a low level position through a pull-down resistor to ensure the accuracy of the circuit output level. At the same time, a capacitor device is designed between the power supply and the ground as a buffer to ensure the stability of the entire circuit operation. The capacitor device is as follows Figure 8 shown.

[0060] Step 2: Assemble the sensor.

[0061] The GPIO interface is used to connect sensor peripheral devices to the core devices of the system. Peripheral devices that use I2C and self-developed protocols include two IoT sensors: water accumulation sensor and temperature and humidity sensor. IoT sensors do not require high-speed data connections, but require stable connections to ensure that the data transmission process is not disturbed. The GPIO interface contains a large number of protocols, and the interface definition of each protocol is different, so the corresponding interface of the protocol needs to be brought out. At the same time, it is necessary to add a protocol logic chip to the self-developed water accumulation sensor to ensure the normal operation of the sensor, so a dedicated expansion version was developed for this system.

[0062] The developed expansion board is connected to the GPIO interface of the core device through a 40PIN interface, and the I2C, SPI, and CAN interfaces are brought out. A protocol logic chip is set on the expansion board so that the self-developed protocol can be connected to the core device through the chip. Each expansion board can connect four sets of water accumulation sensors with self-developed protocols, and the other brought out interfaces provide the possibility of adding other sensors. The GPIO expansion board and the core device are placed in the cabinet, and the other connected sensors are brought out from the cabinet by inserting the corresponding interfaces of the expansion board. The connection diagram of the water accumulation detection sensor is as follows Fig. 9 shown.

[0063] It can be found that each water accumulation detection node can be divided into two polarities, and each polarity can be connected to 9 water accumulation detection nodes. The water accumulation detection node can be activated by connecting water accumulation detection plates of different polarities. The activated water accumulation detection node transmits the signal to the expansion board, and the core device receives the auxiliary judgment signal of water accumulation. Each detection board can connect 4 water accumulation detection nodes, so a system can connect 36 groups of water accumulation detection plates, and any group of systems that detect water accumulation will receive an auxiliary judgment signal of water accumulation.

[0064] Step 3: Design the corresponding PCB and 3D simulation to detect water accumulation targets, and display the water accumulation target detection results on a large screen.

[0065] Altium Designer software was used to design the corresponding PCB layout. Considering the size and volume of the sensor should be as small as possible, the aesthetics of the sensor, and the scalability and flexibility requirements for future upgrades or modifications, the layout and routing of the components in the layout were carefully designed, and finally a PCB layout that can ensure the circuit board has strong performance and efficient operation was obtained.

[0066] The LED screen controller uses the Dahua DH-PHTOC-PT11 control card, which is compatible with the Nova traffic protocol and needs to be developed based on this communication protocol. The traffic protocol has a plug-and-play host computer program, but it is incompatible with the Nvidia Jetson AGX Orin architecture and system used in this system, and needs to be redeveloped. The control card comes with a complete communication protocol source code. After transplantation and testing, the control protocol is successfully connected to the core device used in this system. The information dissemination module is also responsible for the LED screen layout, cutting the overlong character string and displaying it in large fonts on the corresponding LED screen.

[0067] The present invention first designs the water accumulation sensor node circuit, then performs related designs, designs the circuit of the water accumulation sensor detection electrode, and then designs the water accumulation detection circuit and related interfaces to facilitate the connection between the circuit and the physical object, and finally performs target detection and displays it on a large screen, and finally simulates to verify the effectiveness and accuracy of the invention.

[0068] Step 4: Combine hardware and software solutions.

[0069] The nodes of this system include cameras, sensors, edge computing devices, antennas and interactive devices. The interactive device uses a large-area LED bulletin board with strong anti-reflection ability to ensure that the driver can see the interactive content clearly from a distance; the sensor uses self-developed or finished sensors to provide physical sensors and artificial intelligence methods for road infrastructure warning and road abnormality warning, and form redundancy to ensure the normal operation of the warning function; Camera: Use a camera with 1080P resolution and infrared night vision function to provide effective protection for night road detection and curve warning. This system has flexible configuration. Each node can freely combine multiple edge computing devices, cameras, sensors, and interactive devices according to different scales to achieve on-demand configuration. For example, in the curve warning scenario, two groups of cameras and two groups of edge computing devices can be assumed on both sides of the curve, and the sensors can be arranged according to actual conditions.

[0070] In order to achieve the modularity and scalability of the system, this system takes edge computing devices as the core, switches and expansion boards as node switching devices, LED screens, surveillance cameras, water accumulation sensors and temperature and humidity sensors as peripheral devices. All three devices have the ability to be replaced or updated individually to meet flexible deployment in complex usage environments.

[0071] LED screens and surveillance cameras are connected to the switch via RJ - The 45 interface communicates bidirectionally with the edge computing device; the water accumulation sensor, temperature and humidity sensor, IoT sensor peripheral devices are connected to the expansion board, providing the edge computing device with the ability to perceive the surrounding environment, enhancing the accuracy of machine learning judgment, and reducing misjudgments and missed judgments. The switch is connected to each connected device through a network cable, which has the advantages of strong anti-interference ability and long-distance laying. The expansion board is connected to the edge computing device through a dedicated 40PIN cable and placed next to the edge computing device. The connection between the sensor uses the self-developed protocol and the I2C protocol, both of which have a certain anti-interference ability.

[0072] The software indicators are to detect cars, trucks, pedestrians, traffic cones, two-wheelers, three-wheelers, convertibles, night vehicles, dogs, cows, sheep, cracks, ice and snow road obstacles or road anomalies. In order to reasonably allocate the computing power of edge computing devices, the artificial intelligence part uses advanced target detection algorithms to reduce the amount of calculation while ensuring accuracy, so that the delay of output information is further reduced when the power consumption of edge computing devices is reduced. The software framework is divided into four parts: system detection module, Kafka module, system analysis module and system information dissemination module. Among them, the system detection module is responsible for artificial intelligence algorithm target detection, Kafka is the information transmission module, the system analysis module is the post-processing module of the system, including sensor information post-processing and information release post-processing, and the information dissemination module is mainly responsible for the docking of the system main program output and the LED screen.

[0073] 1. System detection module

[0074] The system detection module is an artificial intelligence processing module. It uses DeepStream as the framework and develops a pruning optimization algorithm based on the YOLOV8 network. It optimizes the accuracy and speed of the original network framework so that it can be deployed on the edge computing module for operation.

[0075] 2. Kafka module

[0076] To ensure the stability of the system, the program is divided into several subroutines for execution. The artificial intelligence part is executed by the system detection module, which outputs the detection results to the system analysis module through Kafka for analysis, and then outputs the analysis results. Adding subroutine modules can avoid the main program crash and thus the system crash, and can also make the system more stable.

[0077] A typical Kafka architecture includes several message producers, several node servers, several message consumers, and a ZooKeeper cluster. Kafka manages cluster configuration, elects leaders, and performs rebalance (i.e., consumer load balancing) when consumer groups change through ZooKeeper. Producers use the push mode to publish messages to brokers, and consumers use the pull mode to subscribe to and consume messages from brokers. In this system, Kafka's Producer is the system detection module, and Consumer is the system analysis module. The system detection module outputs the detection result string, which is passed to the system analysis module through Kafka. The protocol layer that communicates with Kafka has been embedded in the system detection module and can be directly connected to Kafka-Server. Before Kafka is started, the front-end process ZooKeeper needs to be started. After ZooKeeper is started, Kafka is pulled up and the message transmission function is started. According to system requirements, a daemon is written. If Kafka stops working, Kafka is restarted to ensure stable operation of the system.

[0078] 3. System analysis module

[0079] This module is a post-processing module, including sensor information post-processing and information release post-processing. Sensor information post-processing includes further weighted processing of sensor information and conditional judgment processing, and information release post-processing includes post-processing of message data transmitted by the system detection module.

[0080] The strings transmitted by the system detection module through Kafka mainly include the result information of the identified objects, as well as the identification information of ice, snow and water accumulation. These strings are all real-time identification information. Due to the restrictions of the screen refresh rate and the output information rules, directly outputting the real-time detection information will result in disorganized information, which will directly affect the curve warning scene and the driver will not be able to obtain effective information. Therefore, the information needs to be cached and processed, and the processed information can be handed over to the information dissemination module and finally displayed on the LED screen.

[0081] Sensor post-processing not only includes comparing and weighting the water accumulation, ice and snow warnings output by the artificial intelligence network with the original data, but also includes a watchdog program to judge whether the sensor transmits error frames or the sensor does not respond. If the number of transmitted error frames or empty frames is greater than 5 times, a reset command is issued to the sensor.

[0082] Information post-release processing is a comprehensive display strategy processing process for various information. If the result string transmitted by the system detection module is legal and weighted with the original sensor data, the result frequency and sensor data parameters appearing in the result are judged, and the result is processed according to the display strategy, and finally output to the information dissemination module in the form of a string and displayed on the LED screen.

[0083] 4. Information dissemination module

[0084] The information dissemination module is mainly responsible for the connection between the system main program output and the LED screen. The LED screen controller uses the Dahua DH-PHTOC-PT11 control card, which is compatible with the Nova traffic protocol and needs to be developed based on this communication protocol. The traffic protocol has a plug-and-play host computer program, but it is incompatible with the Nvidia Jetson AGX Orin architecture and system used in this system, and needs to be redeveloped. The control card comes with a complete communication protocol source code. After transplantation and testing, the control protocol is successfully connected to the core device used in this system. The information dissemination module is also responsible for the LED screen layout, cutting the extra-long character string and displaying it in large font on the corresponding LED screen to remind the drivers of passing vehicles to drive carefully and pass slowly.

[0085] The present invention first designs the water accumulation sensor node circuit, then performs related designs, designs the circuit of the water accumulation sensor detection electrode, and then designs the water accumulation detection circuit and related interfaces to facilitate the connection between the circuit and the physical object, and finally performs target detection and displays it on a large screen, and finally simulates to verify the effectiveness and accuracy of the invention.

[0086] Simulation 1: In order to prevent the impact of various emergencies encountered during the deployment of sensors, and to better evaluate the performance of the system and find problems before formal deployment, the sensor circuit was simulated and tested using Protues simulation software to simulate the real-world operating environment and conditions. The simulation of the circuit minimized the risk of system failure and greatly reduced the cost of trial and error. Fig.10 Shown is a simulation diagram of the sensor node circuit.

[0087] During the circuit simulation, the water accumulation detected by the two electrodes may be unstable in the actual environment. Some water is relatively pure, while some water has more impurities, which will form different resistance values. Therefore, a sliding rheostat is introduced in the simulation process to replace the situation where the two electrodes encounter water. RV2 has different values ​​for different water qualities, and the detection threshold of RV1 is set at the same time. It can be found that after the circuit is powered on, no water is detected, the blue LED_D1 is on, and the red LED_D1 is not on; when water is detected, the blue LED_D1 and the red LED_D1 light up at the same time to achieve the actual effect of detecting water accumulation. Fig.11 This is the simulation circuit diagram of the water accumulation sensor node circuit when it detects water accumulation.

[0088] Simulation 2: With the PCB layout, 3D simulation diagram, and simulation test complete, a water accumulation sensor and GPIO interface expansion board were produced. The water accumulation sensor is connected to the GPIO expansion board, and the results detected by the sensor are transmitted to the edge computing device for processing. The processed results are transmitted to the LED display screen for real-time monitoring of test performance. Simulate water accumulation and no water accumulation, and observe the results displayed on the LED display screen.

[0089] For the mainboard of the water accumulation sensor node, after the system is powered on, when no water accumulation is detected, the blue light on the mainboard of the sensor node is on and the red light is off. At this time, since no water accumulation is detected, the LED display screen only displays temperature information. Then, the detection electrode is directly placed in the water to simulate that the water accumulation reaches a certain amount. At this time, the red light is on to indicate that water accumulation is detected. Based on the above, it can be concluded that the water accumulation sensor of the present invention can effectively identify water accumulation conditions, and can be displayed on the LED screen, and can provide effective information. Prove the effectiveness and rationality of the water accumulation sensor proposed in this patent.

[0090] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modifications to the same effective embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, same change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of the technical solution of the present invention.

Claims

1. The steps of the target detection method based on the sensor large screen display are as follows: Step 1: Install the sensor circuit: In the design of water accumulation sensor, the core is the design of comparator circuit. In this circuit design, a LM393 voltage comparator is first introduced, which can compare the voltage at the same phase terminal and the voltage at the opposite phase terminal, and output the comparison result in the form of high and low levels. A sliding rheostat is connected to the inverting end to adjust the voltage threshold for judging whether there is water accumulation. A water accumulation sensing device is designed at the inverting input end. It consists of two electrodes DET_A and DET_B with opposite polarities placed at a certain distance, one electrode is grounded, and the other is connected to the inverting input end; at the same time, an LED alarm device is connected to the output end of the comparator to judge the high and low levels of the output, high level is on, and low level is off; assuming that the total voltage is 10V, when there is no water between the two electrodes, the resistance composed of the two electrodes DET_A and DET_B is infinite by default. According to the circuit voltage division principle, the voltage divided by the inverting end approaches 10V. At this time, the voltage at the inverting end is greater than the voltage at the reverse end, the output end outputs a high level, and the LED lights up; when there is water between the two electrodes, it is equivalent to having a medium between the two electrodes, which is equivalent to the existence of resistance. The voltage threshold is set by adjusting the sliding rheostat at the inverting end so that the voltage at the inverting end is just greater than the voltage at the inverting end. At this time, the output end outputs a low level and the LED lights up; Since the water accumulation sensor node circuit is designed to detect the absence of water at a high level and the presence of water at a low level, in order to facilitate subsequent work, an SN74LVC1G04DBVR inverter is added to the output end of the circuit to change the high and low levels of the circuit output; In addition, in the design of the circuit, if there is water between the two plates, it will change its structure and change its on-off; but the range of the two plates is too small, so we expand the two plates to 18, DET_A and DET_B are divided into two groups of nine and placed according to the position in Figure 5; Assume that the red electrode piece is DET_A and the blue electrode piece is DET_B, and they are placed on the electrode plate in a cross-spaced manner, and the electrode plates are placed in different positions for water accumulation detection, which increases the detection area; such a circuit integrated with several modules is the designed water accumulation sensor node circuit; no matter where the water drops, as long as any two red and blue electrode pieces with different polarities can be connected, a comparator circuit can be formed to achieve the purpose of detecting water accumulation; after the water accumulation sensor node circuit is designed, four identical sensor nodes are combined together, and an SN74LVC32APWR AND gate logic chip is introduced to perform an AND gate logic operation on the four sensor nodes in pairs, and the output result is performed another AND operation, and finally the signal of the entire water accumulation sensor detecting water accumulation is output; It can be seen that no matter which of the four water accumulation sensor nodes detects the water accumulation signal, it will eventually output the detected water accumulation signal through logical operation; and in such a circuit, when the DO interface of each sensor node has no output, the circuit will be in a floating state between high and low levels. The floating state of DO is pulled to a low level position through a pull-down resistor to the ground, ensuring the accuracy of the circuit output level; at the same time, a capacitor device is designed between the power supply and the ground as a buffer to ensure the stability of the entire circuit operation; Step 2: Assemble the sensor: The GPIO interface is used to connect sensor peripheral devices to the core devices of the system. The peripheral devices that use I2C and self-developed protocols include two IoT sensors: water accumulation sensor and temperature and humidity sensor. IoT sensors do not require high-speed data connections, but require stable connections to ensure that the data transmission process is not disturbed. The GPIO interface contains a large number of protocols, and the interface definition of each protocol is different, so the corresponding interface of the protocol needs to be brought out. At the same time, it is necessary to add a protocol logic chip to the self-developed water accumulation sensor to ensure the normal operation of the sensor, so a dedicated expansion version is developed for this system. The developed expansion board is connected to the GPIO interface of the core device through a 40PIN interface, and the I2C, SPI, and CAN interfaces are brought out. A protocol logic chip is set on the expansion board so that the self-developed protocol can be connected to the core device through the chip; each expansion board can connect four sets of water accumulation sensors with self-developed protocols, and the other brought-out interfaces provide the possibility of adding other sensors; the GPIO expansion board and the core device are placed in the cabinet, and the other connected sensors are brought out from the cabinet by inserting the corresponding interfaces of the expansion board; As can be seen from the figure, each water accumulation detection node can be divided into two polarities, and each polarity can be connected to 9 water accumulation detection nodes; connecting water accumulation detection plates of different polarities can activate the water accumulation detection node, and the activated water accumulation detection node transmits the signal to the expansion board, at which time the core device receives an auxiliary judgment signal of water accumulation; each detection board can be connected to 4 water accumulation detection nodes, so a system can be connected to 36 groups of water accumulation detection plates, and any group of systems that detect water accumulation will receive an auxiliary judgment signal of water accumulation; Step 3: Design the corresponding PCB and 3D simulation to detect water accumulation targets, and display the water accumulation target detection results on a large screen: Altium Designer software was used to design the corresponding PCB layout. Considering the size and volume of the sensor should be as small as possible, the aesthetics of the sensor, and the scalability and flexibility requirements for future upgrades or modifications, the layout and routing of the components in the layout were carefully designed, and finally a PCB layout that can ensure the circuit board has powerful performance and efficient operation was obtained. The LED screen controller uses the Dahua DH-PHTOC-PT11 control card, which is compatible with the Nova traffic protocol and needs to be developed based on this communication protocol; the traffic protocol has a plug-and-play host computer program, but it is incompatible with the core device Nvidia Jetson AGX Orin architecture and system used in this system and needs to be redeveloped; the control card comes with a complete communication protocol source code. After transplantation and testing, the control protocol is successfully connected to the core device used in this system; The information dissemination module is also responsible for the layout of the LED screen, cutting the extra-long character string and displaying it in large font on the corresponding LED screen; Step 4: Combination of hardware and software solutions: The nodes of this system include cameras, sensors, edge computing devices, antennas and interactive devices. The interactive devices use large-area LED bulletin boards with strong anti-reflection capabilities to ensure that drivers can see the interactive content clearly from a distance. The sensors use self-developed or finished products, and provide physical sensors and artificial intelligence methods for road infrastructure warnings and road anomaly warnings. Cross-validation and redundancy are formed to ensure the normal operation of the warning function. Camera: Use a camera with 1080P resolution and infrared night vision function to provide effective protection for nighttime road detection and curve warning. The system has flexible configuration. Each node can freely combine multiple edge computing devices, cameras, sensors, and interactive devices according to different scales to achieve on-demand configuration. For example, in a curve warning scenario, two groups of cameras and two groups of edge computing devices can be assumed on both sides of the curve, and sensors can be arranged according to actual conditions. In order to realize the modularity and scalability of the system, this system takes edge computing devices as the core, switches and expansion boards as node switching devices, LED screens, surveillance cameras, water accumulation sensors and temperature and humidity sensors as peripheral devices. All three devices have the ability to be replaced or updated separately to meet the flexible deployment requirements in complex usage environments. The LED screen and surveillance camera are connected to the switch, and communicate with the edge computing device in two directions through the RJ-45 interface; the water accumulation sensor, temperature and humidity sensor, and IoT sensor peripherals are connected to the expansion board to provide the edge computing device with the ability to perceive the surrounding environment, enhance the accuracy of machine learning judgment, and reduce misjudgments and missed judgments; the switch is connected to each connected device through a network cable, which has the advantages of strong anti-interference ability and long-distance laying. The expansion board is connected to the edge computing device through a dedicated 40PIN cable and placed next to the edge computing device. The connection between the sensor uses the self-developed protocol and the I2C protocol, both of which have a certain degree of anti-interference ability; The software indicators are to detect cars, trucks, pedestrians, traffic cones, two-wheelers, three-wheelers, convertibles, night vehicles, dogs, cows, sheep, cracks, ice and snow road obstacles or road anomalies; in order to reasonably allocate the computing power of edge computing devices, the artificial intelligence part uses advanced target detection algorithms to reduce the amount of calculation while ensuring accuracy, so that the delay of output information is further reduced when the power consumption of edge computing devices is reduced; the software framework is divided into four parts: system detection module, Kafka module, system analysis module and system information dissemination module; the system detection module is responsible for artificial intelligence algorithm target detection, Kafka is the information transmission module, the system analysis module is the post-processing module of the system, including sensor information post-processing and information release post-processing, and the information dissemination module is mainly responsible for the connection between the system main program output and the LED screen; 1. System detection module The system detection module is an artificial intelligence processing module. It uses DeepStream as the framework and develops a pruning optimization algorithm based on the YOLOV8 network. It optimizes the accuracy and speed of the original network framework so that it can be deployed on the edge computing module.

2. Kafka module To ensure the stability of the system, the program is divided into several subroutines for execution; the artificial intelligence part is executed by the system detection module, which outputs the detection results to the system analysis module through Kafka for analysis, and then outputs the analysis results; adding subroutine modules can avoid the main program crash and thus the system crash, and can also make the system more stable; A typical Kafka architecture includes several message producers, several node servers, several message consumers, and a ZooKeeper cluster. Kafka manages cluster configuration, elects leaders, and performs rebalance (i.e., consumer load balancing) when consumer groups change through ZooKeeper. Producers use the push mode to publish messages to brokers, and consumers use the pull mode to subscribe to and consume messages from brokers. In this system, Kafka's Producer is the system detection module, and Consumer is the system analysis module. The system detection module outputs the detection result string, which is passed to the system analysis module through Kafka. The protocol layer that communicates with Kafka has been embedded in the system detection module and can be directly connected to Kafka-Server. Before Kafka is started, the front-end process ZooKeeper needs to be started. After ZooKeeper is started, Kafka is pulled up to start the message transmission function. According to system requirements, a daemon is written. If Kafka stops working, Kafka is restarted to ensure stable operation of the system.

3. System analysis module This module is a post-processing module, including sensor information post-processing and information release post-processing; sensor information post-processing includes further weighted processing of sensor information and conditional judgment processing, and information release post-processing includes post-processing of message data transmitted by the system detection module; The strings transmitted by the system detection module through Kafka mainly include the result information of the identified objects, as well as the identification information of ice, snow and water accumulation. These strings are all real-time identification information. Due to the restrictions of screen refresh rate and output information rules, directly outputting real-time detection information will result in disorganized information, which will directly affect the curve warning scene and the driver will not be able to obtain effective information. Therefore, the information needs to be cached and processed, and the processed information can be handed over to the information dissemination module and finally displayed on the LED screen; The sensor post-processing includes not only the comparison and weighting of the water accumulation, ice and snow warning output by the artificial intelligence network and the original data, but also includes a watchdog program to judge the sensor transmission error frame or sensor no response. If the error frame or empty frame transmitted is more than 5 times, a reset command is issued to the sensor. Information post-release processing is the process of display strategy processing of various information; If the result string transmitted by the system detection module is legal and weighted with the original data of the sensor, the result frequency and sensor data parameters appearing in the result are determined, and the result is processed according to the display strategy, and finally output to the information dissemination module in the form of a string and displayed on the LED screen; 4. Information dissemination module The information dissemination module is mainly responsible for the connection between the system main program output and the LED screen; The LED screen controller uses the Dahua DH-PHTOC-PT11 control card, which is compatible with the Nova traffic protocol and needs to be developed based on this communication protocol; the traffic protocol has a plug-and-play host computer program, but it is incompatible with the core device Nvidia Jetson AGX Orin architecture and system used in this system and needs to be redeveloped; the control card comes with a complete communication protocol source code. After transplantation and testing, the control protocol is successfully connected to the core device used in this system; The information dissemination module is also responsible for the layout of the LED screen. It cuts extra-long character strings and displays them in large fonts on the corresponding LED screen to remind drivers of passing vehicles to drive carefully and pass slowly.

Images