A traffic light control method based on environmental edge computing

Abstract

The present invention proposes a traffic light control method based on environmental edge computing, which belongs to the field of edge computing technology. The method includes the following steps: the data center part discovers a mobile device that can serve as an edge server; the camera starts to collect video frames, and the data center part performs initial target detection; the target detection result is sent to the integration manager; the data center part crops and fills the video frame to create a video subframe; then encodes it into a byte stream; the data center part allocates an edge server; and sends the transmission data to the selected edge server; the edge computing part receives and decodes the transmission data, performs target detection, and sends the target detection result to the data center part; the integrated target detection result of the data center part obtains the final target detection result, and controls the traffic light according to the final result. The present invention unloads the video frame to the edge mobile device in a lightweight and opportunistic manner, and uses it to control the traffic light in real time.

Description

Technical Field

[0001] The present invention belongs to the fields of Internet of Things technology and edge computing technology, and in particular to a traffic light control method based on environmental edge computing. Background Art

[0002] In the 21st century, with the rapid development of economy and technology, innovative mobile applications and services such as augmented reality, federated learning, object detection and edge caching are increasingly emerging. These applications have greatly increased the demand for computing resources, which are usually provided by cloud servers. However, due to the heavy load and long distance of the backhaul link, the communication delay is high. For traffic control services, the acceptable delay time ranges from 10 milliseconds to 100 milliseconds.

[0003] Edge computing is considered to be complementary to cloud computing and tends to provide elastic resources at the edge of the network. To reduce communication latency, edge devices that are physically closer to the network than the cloud infrastructure are used. These edge devices are called edge servers. EdgeGo is a mobile resource sharing framework that uses mobile edge servers (mounted on mobile robots or drones) to enable low-cost deployment of 6G edge computing. Unfortunately, the framework does not fully utilize the rich processing power of increasingly powerful ambient devices such as smartphones. MiVeCC is a vehicle-edge-cloud simulation framework designed to optimize vehicle cooperative control in large areas with multiple intersections. However, it does not consider resource budgets and the impact of communication and computation changes.

[0004] In smart cities, resource-constrained terminal devices can enhance various AI services by connecting to edge servers. However, deploying dedicated heavyweight edge servers and resources to establish large-scale service coverage can be very expensive and require more manpower, material and financial resources. The deployment of traditional edge servers requires the purchase of physical equipment, site leasing, installation and maintenance costs, additional costs for electricity and backhaul transmission, expensive capital and operating expenses for base stations, and base station deployment usually requires a long deployment cycle. Summary of the invention

[0005] The purpose of the present invention is to provide a traffic light control method based on environmental edge computing to solve the technical problems of high cost and long deployment cycle of mobile edge devices for traffic light control.

[0006] In order to solve the above technical problems, the traffic light control method based on environmental edge computing of the present invention comprises the following steps:

[0007] Step S1, the data center part periodically broadcasts a WiFi beacon embedded with a request service through an access point, and discovers a mobile device that can serve as an edge server through a service manager;

[0008] Step S2: The camera starts to collect video frames, and each frame is transmitted through two paths; one is used for local detection by the local detector in the data center part, and the other is used for remote detection in the edge computing part;

[0009] Step S3: The data center runs the YOLOv5s network detection model on the local detector, performs analysis of the video frame in the local area with limited resources, and performs initial target detection, i.e., target positioning and classification; divides the video frame into boundary lines, extracts the targets crossing the boundary lines according to the coordinate position information of the detected targets, and sends the index of the frame where the target is located and the target detection result to the integration manager;

[0010] The target detection result includes a detection box, a label and a score, which are directly obtained after the local detection model and the remote detection model detect the video frame; the detection box refers to the bounding box of the detected target, which is used to describe the position and size of the target in the video frame; the label refers to the category to which the detected target belongs; and the score refers to the probability of belonging to the category, that is, the confidence level;

[0011] Step S4: The cropping manager of the data center part crops each video frame into a number of main slices according to the boundary lines; then, fills each main slice with surrounding filling slices to create a new video subframe; and then encodes it into a byte stream that can be used for transmission;

[0012] Step S5: The offload manager of the data center part allocates a corresponding edge server to each video subframe through a redundant server selection mechanism according to the CPU load and RSSI information of each edge server transmitted by the service manager; and assembles the IP, port and encoded video subframe corresponding to the edge server into transmission data and stores it in a sending queue, waiting to be sent to the selected edge server;

[0013] The redundant server selection mechanism is a mechanism that allocates each video subframe to multiple edge servers for processing instead of just a single edge server. Because the computing resources of edge servers are limited and the communication instability caused by the movement of edge servers will make the task reliability low, more edge servers need to be selected to process the same task, that is, redundant servers are selected.

[0014] Step S6: The edge server of the edge computing part receives the transmission data sent by the data center part, decodes the transmission data in the receiving queue, performs target detection through the Nanodet-Plus target detection model of the remote detector, and sends the current video subframe index and target detection result obtained by the target detection to the data center part;

[0015] Step S7: The integration manager of the data center integrates the target detection results obtained by local detection and the target detection results obtained by remote detection of the edge server to obtain the final target detection result, and controls the traffic lights according to the final target detection result.

[0016] Furthermore, in step S1, the data center part embeds service information in the beacon to enable the data center part to efficiently discover mobile devices that can provide the requested service; nearby mobile devices perform service queries by parsing the beacon information; if the mobile device can provide the service requested by the data center part, it will actively establish a WiFi connection with the data center part and become an edge server.

[0017] Furthermore, the data center part has a Bloom filter and provides a list of requested services; each edge device has a Bloom filter and provides a list of available services; in the service map of the data center part, the bit array Initially, it is set to an array of all 0s; then the data center part hashes the name of each requested service in the list of requested services, maps it to the corresponding position through multiple hash functions, and converts the bit array Set the bits corresponding to these positions in the bit array to 1; then Embedded in beacons, beacons are broadcast periodically; in the service query of the edge computing part, each edge device parses the received beacon to obtain the bit array , and query the list of services it can provide; the edge device performs hash mapping on the name of each service it can provide, and the mapping result is the same as the received bit array Compare, the bit with mapping result of 1 is in the bit array Only when the corresponding bits in the data center are all 1, the edge device has the service requested by the data center; if the edge device has at least one service that meets the requirements, that is, at least one service that the edge device can provide matches the service requested by the data center, then the edge device is connected to the wireless network of the data center and becomes an edge server.

[0018] Furthermore, the edge server sends a network connection request to the data center part. After the connection is successful, the edge server registers the device status and service information with the data center part; the data center part initiates a service request to the edge computing part according to the edge offloading needs, and the edge computing part processes the request and returns the result.

[0019] Furthermore, the data center part predetermines the padding size before cropping each video frame into several main slices. Since the edge server performs object detection on the cropped video frame, i.e., the video subframe, and cropping may split a target instance into multiple erroneous instances, it is necessary to determine the minimum detectable target width. and minimum detectable target height To calculate the padding size, padding the corresponding frame part on each main slice can reduce the number of error instances. Specifically, the target category to be detected is recorded as , including people, bicycles, cars, motorcycles, buses, and trucks; for each category , the minimum width and minimum height that the local detector can detect are recorded as ,in Indicates category The minimum width of Indicates category The minimum height of Represents the minimum width and minimum height of all categories. Indicates the minimum width of all categories. Represents the minimum height of all categories; the width threshold and height threshold of small objects are determined by selecting the maximum value of the product of the minimum width and the minimum height among all categories, denoted as The padding width is then determined by halving the width and height thresholds for small objects. and fill height ,Right now ;

[0020] The data center evenly divides the complete video frame into multiple main slices according to the available edge servers. The width and height of each main slice are recorded as and ,Right now ;in Represents the index of each main slice, represents the number of edge servers, and Represents the width and height of a complete video frame; in addition, each main slice will be resized accordingly after padding to form a video subframe.

[0021] Furthermore, the filling rule of each main piece is as follows: each main piece contains two edge lines of width and two edge lines of height. If its edge line does not overlap with the edge line of the complete frame, the edge line is filled; if a main piece is filled once in the width direction, the adjusted width is: Similarly, if the height is filled once, the adjusted height is: .

[0022] Furthermore, in step S5, the offload manager can not only speed up the detection process, but also ensure the reliability and low cost of the task; for each subtask / video subframe, the cost of each edge server is estimated based on the end-to-end delay and payment; then, the edge server with the lowest cost and meeting the deadline requirement is selected; if no edge server meets the deadline requirement, the edge server with the smallest end-to-end delay is selected to ensure that the current subtask is not missed; after selecting the first optimal edge server for each subtask, decide whether to continue to select more edge servers based on the reliability of the current subtask.

[0023] Furthermore, each time a video frame is generated, it is divided into video subframes, and the video subframe set is ;have The edge servers that can provide services are connected to the data center part, and their collection is ;use Represents a video subframe Is it assigned to the edge server? ;

[0024] Video subframe On edge servers The computation time on is:

[0025]

[0026] in, The number of CPU cycles required for decoding; The number of CPU cycles required for detection; Represents edge server CPU load; is a The calculation time function of For edge servers CPU frequency;

[0027] Data center to edge server The communication time is:

[0028]

[0029] in, Indicates the size of the input data; From data center to edge server Data transfer rate; Represents edge server The download time of the test result data to the data center is relatively small compared to the input data, so the download time is Set to a fixed constant value based on the round-trip time (RTT);

[0030] Video subframe On edge servers The processing delay on is calculated as:

[0031] .

[0032] Furthermore, the video subframe On edge servers Payment Calculated as:

[0033]

[0034] in, Indicates unit payment;

[0035] On edge servers Cost Calculated as:

[0036]

[0037] in, and are the weighting coefficients of payment and delay, respectively;

[0038] Total cost of the current frame It can be calculated as:

[0039]

[0040] Furthermore, after selecting the first optimal edge server for each subtask, deciding whether to continue selecting more edge servers is implemented according to the reliability of the current subtask in the following manner: if the reliability of the subtask is lower than the task reliability threshold, redundant edge servers are selected one by one until the reliability requirement is met; then the edge server set excludes the selected edge servers and is arranged in ascending order of cost to select the redundant edge servers of the subtask; whenever deciding whether to select an edge server, the edge server reliability must be greater than a minimum threshold to avoid selecting too many redundant edge servers; at the same time, the deadline requirement must be met; and in order to avoid queue delays, each edge server cannot be selected twice in a set of video subframes.

[0041] Further, step S7 specifically includes: all target detection results from the data center part and the target detection results returned by the edge server are collected as initial results and then added to the detection list; targets that do not cross any boundary lines in the complete video frame will be deleted from the detection list; if the intersection of the detection boxes of the targets between the video subframes exceeds 75%, it means that the same target may be repeatedly detected by multiple edge servers, then the one with the higher score is accepted as a better choice, that is, the target with the high score continues to remain in the detection list, and the label corresponding to the score is used as the category of the target, while the other target with a lower score is deleted; finally, if the intersection of the detection boxes of the target crossing the boundary line in the complete video frame and the target between the video subframes exceeds 75%, it means that the same target may be repeatedly detected by the local and edge servers. Since the target may be incomplete when input to the edge server, resulting in edge server detection errors, the one in the complete frame is considered better and the other is deleted.

[0042] Compared with the prior art, the present invention has the following beneficial technical effects: instead of pre-deploying base stations and servers, the present invention offloads the slice-level target detection task to the devices owned by the crowd to realize intelligent traffic light control. The main work of the present invention is how to discover available devices and how to offload tasks. Among them, the access point will periodically send beacons embedded with the required services to discover available devices as edge servers. The detection model is run locally for initial target detection. Then the targets across any boundaries are extracted. In order to improve the detection accuracy and shorten the decoding time, each frame is cropped into multiple slices for fine-grained detection on the selected server. The effective redundant server mechanism ensures the reliability of task completion. In addition, the CPU load of the server is used to estimate its computing power and the computing power is used to estimate the resource payment. Finally, the local and remote detected targets are integrated to obtain the final target detection result, which provides a basis for the development of a low-cost and reliable system. Through the proposed lightweight task splitting, redundant edge selection and fast service discovery scheme, the present invention can offload video frames to edge mobile devices in a lightweight and opportunistic manner and use them to control traffic lights in real time.

[0043] The crowd-assisted system of edge servers found in the present invention is very flexible and can be quickly expanded as needed. Finally, frequent disasters such as heavy rains, landslides, forest fires, and earthquakes may damage the network infrastructure, resulting in no end-to-end communication network in the area, but the opportunistic network provisioning assisted by the crowd in the present invention can ensure post-disaster availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0045] Figure 1 It is a structural schematic diagram of the traffic light control system based on environmental edge computing of the present invention.

[0046] Figure 2 It is a structural schematic diagram of the connectionless service discovery based on Bloom filter and WiFi beacon of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Example 1

[0049] like Figure 1 As shown, the present invention provides a traffic light control system with ambient edge intelligence based on edge computing. Edge computing is considered to be an important means to realize intelligence anywhere in a smart city, where resource-constrained terminal devices can enhance various AI services by connecting to edge servers. However, deploying dedicated heavyweight edge servers and resources to establish large-scale service coverage can be very expensive, requiring more manpower, material and financial resources. Therefore, the mobile devices carried by mobile users can be used as lightweight edge servers, which have great potential to provide computing resources, which may make up for the shortcomings of dedicated edge servers.

[0050] This paper focuses on the emerging smart city application of intelligent traffic light control. Traffic cameras control traffic lights through real-time detected traffic information, which causes a large amount of image processing workload in vast urban areas. Based on the idea of ​​ambient edge intelligence, a new control system is proposed to achieve collaboration among cameras, traffic lights and mobile devices. Through the proposed lightweight task splitting, redundant edge selection and fast service discovery scheme, video frames can be offloaded to edge mobile devices in a lightweight and opportunistic manner and used to control traffic lights in real time.

[0051] The control system includes a data center part and an edge computing part. The data center part includes: a camera, a local detector, an integration manager, a service manager with a Bloom filter, a clipping manager and an offloading manager, and the edge computing part includes a Bloom filter, a decoder and a remote detector of an edge node. Figure 1 The solid arrows represent wired data transmission, and the dashed arrows represent wireless data transmission. The edge computing part actually includes multiple devices, and the ellipsis indicates that there are more than two devices.

[0052] The present invention proposes a traffic light control method based on environmental edge computing, comprising the following steps:

[0053] Step S1: The data center periodically broadcasts WiFi beacons embedded with service requests through access points, and discovers mobile devices that can serve as edge servers through a service manager.

[0054] Each edge server registers with the service manager after becoming a member of the server set and periodically reports its device status through heartbeat packets. Specifically, the device status includes IP, CPU load, RSSI (received signal strength indicator) and service information (list of available services and ports).

[0055] Step S2: The camera starts to collect video frames, and each frame will be transmitted through two paths; one is used for local detection by the local detector in the data center part, and the other is used for remote detection in the edge computing part.

[0056] Specifically, the local detection selects the YOLOv5s network model as the detection model to obtain rough target information on the complete frame, that is, only obtain target information across the boundary line. The remote detection selects the lightweight Nanodet-Plus target detection model to obtain more accurate target information on the video subframe, that is, to further obtain other potential targets in the data center part that are not detected or detected inaccurately by the local detection.

[0057] Step S3: The data center runs the YOLOv5s network detection model on the local detector, performs analysis of the video frame in the local with limited resources, and performs initial target detection, i.e., target location and classification. The video frame is divided into boundary lines, and the targets crossing the boundary lines are extracted based on the coordinate position information of the detected targets, and the index of the frame where the targets are located and the target detection results are sent to the integration manager. The targets include but are not limited to people, bicycles, cars, motorcycles, buses, and trucks.

[0058] The target detection results include detection boxes, labels and scores, which are directly obtained after the local detection model and the remote detection model detect the video frame; the detection box refers to the bounding box of the detected target, which is used to describe the position and size of the target in the video frame; the label refers to the category to which the detected target belongs, i.e., people, bicycles, cars, motorcycles, buses and trucks, etc.; the score refers to the probability of belonging to the category, i.e., confidence.

[0059] Step S4: The cropping manager in the data center part crops each video frame into several main slices according to the boundary lines; then, each main slice is filled with surrounding filling slices to create a new video subframe; and then encoded into a byte stream that can be used for transmission; by weakening the dependency between slices, the parallelism of slice-level detection is achieved.

[0060] Step S5: The offload manager of the data center allocates a corresponding edge server to each video subframe through a redundant server selection mechanism based on the CPU load and RSSI information of each edge server transmitted by the service manager. The IP, port and encoded video subframe corresponding to the edge server are assembled into transmission data and stored in a sending queue, waiting to be sent to the selected edge server.

[0061] The redundant server selection mechanism is a mechanism that allocates each video subframe to multiple edge servers for processing instead of just a single edge server. Because the computing resources of edge servers are limited and the communication instability caused by the movement of edge servers will make the task reliability low, more edge servers need to be selected to process the same task, that is, redundant servers are selected.

[0062] Step S6: The edge server of the edge computing part receives the transmission data sent by the data center part, decodes the transmission data in the receiving queue, performs target detection through the Nanodet-Plus target detection model of the remote detector, and sends the current video subframe index and target detection result obtained by the target detection to the data center part; the target detection result includes the detection box, label and score.

[0063] Step S7: The integration manager of the data center integrates the target detection results obtained by local detection and the target detection results obtained by remote detection of the edge server to obtain the final target detection result, and controls the traffic lights according to the final target detection result.

[0064] Optionally, the red-green light switching time of the traffic light is adaptively adjusted according to the target number of the current video frame, thereby ensuring the safety and smoothness of traffic.

[0065] Example 2

[0066] like Figure 2As shown in the figure, a connectionless service discovery protocol based on Bloom filter and WiFi beacon is used. It allows edge devices to know whether they have the service requested by the data center part even when they are not connected to the network of the data center part. The traditional method requires the data center part to establish a communication connection (such as WiFi) with the devices within the coverage area to discover available devices. In densely populated areas, the number of devices around is huge. If the data center part frequently communicates with all devices, or even establishes communication connections with some devices without services, it will waste energy and time.

[0067] The WiFi protocol has a feature that even in the absence of data transmission, beacons must be broadcast periodically (usually 100 milliseconds) to announce the existence of its own network, and nearby devices can connect to this network. The present invention utilizes this feature of the WiFi protocol. The data center part embeds service information in the beacon to allow the data center part to efficiently discover mobile devices that can provide the requested service. Nearby mobile devices perform service queries by parsing the beacon information. If the mobile device can provide the service requested by the data center part, it will actively establish a WiFi connection with the data center part and become an edge server. There is almost no additional communication overhead in the service discovery process.

[0068] Use bloom filter optimization method in service manager.

[0069] Standard Bloom filter: First, all elements of the set to be mapped are traversed, and then multiple independent hash functions are used to map all elements to the array in turn. The element is mapped to different positions / indexes in the set and the bits at the corresponding positions are set to 1. Then when you need to query an element, if the corresponding bit at any position of the element after mapping is 0, then the element is definitely not in the set to be mapped. Otherwise, if the corresponding bits at all positions of the element after mapping are 1, then either the element is in the set, or the bits corresponding to these positions happened to be set to 1 when mapping another element (resulting in a false positive rate of 1). The number of elements to be mapped The false positive rate of the Bloom filter is determined by The bit array size can be , Number of hash functions To optimize. Then The calculation formula is:

[0070] .

[0071] Service discovery optimization: Apply the standard Bloom filter described above to the service discovery process. Figure 2In order to more intuitively describe the internal processing mechanism of the Bloom filter, the small dotted line box on the left indicates that service mapping is the processing mechanism of the Bloom filter in the data center part. The large dotted line box on the left indicates the specific processing process of service mapping. In order to distinguish the mapping process of different services, the dotted arrows in the large dotted line box on the left indicate the mapping process for a certain service, and the solid arrows in the large dotted line box on the left indicate the mapping process for another service. Similarly, the small dotted line box on the right indicates that service query is the processing mechanism of the Bloom filter in the edge part. The large dotted line box on the right indicates the specific processing process of service query. In order to distinguish the query process of different services, the dotted arrows in the large dotted line box on the right indicate the query process for a certain service, and the solid arrows in the large dotted line box on the right indicate the query process for another service.

[0072] The data center part has a Bloom filter and provides a list of requested services (the set to be mapped). Each edge device has a Bloom filter and provides a list of available services (the set to be queried). In the service mapping of the data center part, the bit array Initially, it is set to an array of all 0s. Then the data center part hashes the name of each requested service in the list of requested services, maps it to the corresponding position through multiple hash functions (such as 3 hash functions), and converts the bit array Set the bits corresponding to these positions in the array to 1 (e.g., "Image_detection" is mapped to the corresponding positions 1, 7, and 11 of the array through hash 1, hash 2, and hash 3, and the bits corresponding to these positions are set to 1; similarly, "Pose_estimation" is mapped to the corresponding positions 2, 7, and 10 of the array, and the bits corresponding to these positions are set to 1). Then, set the bit array Embedded in the beacon. In the service query of the edge computing part, each edge device parses the received beacon to obtain the bit array , and query the list of services it can provide.

[0073] Specifically, taking an edge device as an example, hash mapping is performed on the name of each service that it can provide (e.g., "Image_detection" is mapped to new positions 1, 7, and 11 through the same hash function including hash 1, hash 2, and hash 3; similarly, "Image_smoothing" is mapped to new positions 3, 7, and 10). The mapping result is the same as the received bit array. Compare, the bit with mapping result of 1 is in the bit array Only when the corresponding bits in are all 1, the edge device has the service requested by the data center.

[0074] For example, the positions corresponding to "Image_detection" in the service query in the edge computing part are 1, 7, and 11, which are the bit arrays sent from the data center part. The values ​​of "Image_smoothing" in the edge computing part are all 1, indicating that the service "Image_detection" that the edge device can provide is the service "Image_detection" requested by the data center, that is, the service that the edge device can provide matches the service requested by the data center. The positions corresponding to "Image_smoothing" in the service query in the edge computing part are 3, 7, and 10, although positions 7 and 10 are sent from the data center. The corresponding bit in is 1, but position 3 is sent from the data center part The corresponding bit in is 0, that is, not all bits corresponding to all positions are 1, indicating that the service "Image_smoothing" that the edge device can provide is not the service requested by the data center, that is, the service that the edge device can provide does not match the service requested by the data center.

[0075] If at least one of the services provided by the edge device meets the requirements, that is, at least one of the services provided by the edge device matches the service requested by the data center part, then the edge device connects to the wireless network of the data center part and becomes an edge server. The edge server sends a network connection request to the data center part (including probe request, probe response, authentication request, authentication response, association request, association response, EAPol, DHCP and other processes in WiFi). After the connection is successful, the edge server registers the device status and service information with the data center part. The data center part initiates a service request to the edge computing part according to the edge offloading needs, and the edge computing part processes the request and returns the result.

[0076] The time complexity of a Bloom filter query is , while the time complexity of traditional linear query is This method can effectively avoid the data center part from communicating with each device individually. Especially when there are many devices in the environment, the communication cost can be effectively reduced.

[0077] Example 3

[0078] The present invention provides a lightweight task splitting and result integration mechanism for target detection. Specifically, it includes predetermining the filling size, slice-level cropping and result integration process. The predetermination of the filling size requires that it be determined by technicians or researchers according to the application scenario before the system is deployed, and the slice-level cropping and result integration are automatically run after the system is deployed. Due to the low computing power of edge servers and limited available resources, the deployment of edge-based deep neural networks remains challenging. A single resource-constrained edge server cannot meet high service quality requirements. Therefore, the data center part must effectively divide the task into multiple subtasks to improve parallelism. However, simple division may lead to a decrease in detection accuracy.

[0079] In order to improve the detection accuracy, we crop the video frame into multiple video subframes, so that the compression rate of the video frame is smaller after it is input into the detection model, so that the resolution is higher during detection. In order to improve the processing parallelism, the present invention distributes these video subframes to multiple edge servers for simultaneous processing. However, when performing target detection on the cropped video subframes, a target may be cropped into multiple incomplete instances, resulting in an edge server being unable to accurately detect.

[0080] In the prior art, the edge device collaboration framework EdgeDuet uses local detectors to extract large and medium-sized targets, and remote detectors to extract small targets that are not detected or detected inaccurately. However, in EdgeDuet, the remote detector is heavyweight, and although it has high accuracy, it is more expensive. The present invention adopts the lightweight Nanodet-Plus target detection model, which can not only detect small targets, but also some large and medium-sized targets. The video resolution input by the remote detector is higher, so the target is more accurately detected by the remote detector. For targets that cross the boundary line, it is more accurate to detect them by the local detector, because these targets are completely input into the local detector, and the targets input into the remote detector may be incomplete. Therefore, the present invention needs to adopt a result integration method to process the detection results.

[0081] Predetermine the padding size: In order to ensure that small objects that cannot be detected by the local detector still have a chance to be detected by the remote detector, it is necessary to determine the minimum object size of each category that the local detector can detect. In general, as the object gets closer to the camera's field of view, its size relative to the frame increases, and vice versa. Therefore, it is possible to calculate the threshold of the object size that the local detector may not detect. To this end, first use the heavyweight detection model to pre-detect some video frames (these video frames are only used to estimate the padding size before the system is deployed. Once the padding size is determined, it is set to a fixed value and the heavyweight detection model is not required after the system is deployed), and save the detection results to the true result list. Then use the local lightweight detection model in the data center part to detect the same video frames. If there is an object information in the true result list and it is not detected by the local detector in the data center part, the object is considered to be a small object. Finally, the padding size is determined based on the maximum size of all small objects obtained.

[0082] Since the edge server performs object detection on the cropped video frame, that is, the video subframe, and cropping may split a target instance into multiple erroneous instances, it is necessary to determine the minimum detectable target width and minimum detectable target height To calculate the padding size, padding the corresponding frame part on each main slice can reduce the number of error instances. Specifically, the target category to be detected is recorded as , including people, bicycles, cars, motorcycles, buses, and trucks; for each category , the minimum width and minimum height that the local detector can detect are recorded as ,in Indicates category The minimum width of Indicates category The minimum height of Represents the minimum width and minimum height of all categories. Indicates the minimum width of all categories. Represents the minimum height of all categories; the width threshold and height threshold of small objects are determined by selecting the maximum value of the product of the minimum width and the minimum height among all categories, denoted as The padding width is then determined by halving the width and height thresholds for small objects. and fill height ,Right now ;

[0083] Slice-level cropping: The data center evenly divides the complete video frame into multiple main slices based on the available edge servers. The width and height of each main slice are recorded as and ,Right now ;in Represents the index of each main slice, represents the number of edge servers, and Represents the width and height of a complete video frame; in addition, each main slice will be resized accordingly after padding to form a video subframe.

[0084] The filling rule for each main piece is as follows: Each main piece contains two edge lines of width and two edge lines of height. If its edge line does not overlap with the edge line of the complete frame, the edge line is filled. If a main piece is filled once in the width direction, the adjusted width is: Similarly, if the height is filled once, the adjusted height is: .

[0085] Result integration: All target detection results from the data center part and the target detection results returned by the edge server are collected as initial results and then added to the detection list. Targets that do not cross any boundary lines in the complete video frame will be deleted from the detection list. If the intersection of the detection boxes of the target between the video subframes exceeds 75%, it means that the same target may be repeatedly detected by multiple edge servers, and the one with the higher score is accepted as a better choice, that is, the target with the high score continues to remain in the detection list, and the label corresponding to the score is used as the category of the target, while the other target with a lower score is deleted; finally, if the intersection of the detection boxes of the target crossing the boundary line in the complete video frame and the target between the video subframes exceeds 75%, it means that the same target may be repeatedly detected by the local and edge servers. Since the target may be incomplete when input to the edge server, resulting in an error in the edge server detection, the one in the complete frame is considered better and the other is deleted.

[0086] Example 4

[0087] The present invention provides a redundant server selection mechanism for task offloading, taking into account that changes in CPU load will lead to changes in computing performance and cost. The redundant server selection mechanism is a mechanism that allocates each video subframe to multiple edge servers for processing rather than just to a single edge server for processing. Because the computing resources of the edge server are limited and the communication instability caused by the movement of the edge server will make the task reliability low, it is necessary to select more edge servers to process the same task, that is, to select redundant servers.

[0088] As the CPU load increases, the decoding and detection time per frame can increase from tens of milliseconds to hundreds of milliseconds, which may make the completion of the task unreliable. In addition, a certain fee needs to be paid for the edge servers to participate to increase their enthusiasm for participation, otherwise users with such mobile devices are unwilling to join. That is, the more abundant the device resources, the higher the payment fee. For edge servers, the lower the CPU load, the lower the latency, but the higher the resource payment. Therefore, when selecting edge servers, an effective task offloading plan must be formulated and a trade-off between performance and payment must be made. The offloading manager can not only be used to accelerate the detection process, but also ensure the reliability and low cost of the task to complete real-time tasks.

[0089] The redundant server selection mechanism is implemented as follows: For each subtask / video subframe, the cost of each edge server is estimated based on the end-to-end delay and payment. Then, the edge server with the lowest cost and meeting the deadline requirement is selected. If no edge server meets the deadline requirement, the edge server with the smallest end-to-end delay is selected to ensure that the current subtask is not missed. After selecting the first optimal edge server for each subtask, it is decided whether to continue to select more edge servers based on the reliability of the current subtask.

[0090] An edge system consisting of multiple edge servers. Whenever a video frame is generated, it is divided into video subframes, and the video subframe set is .have The edge servers that can provide services are connected to the data center part, and their collection is .use Represents a video subframe Is it assigned to the edge server? Therefore, the data center part needs to determine the edge server selection for each subtask through the following delay analysis, payment analysis, and cost analysis to minimize the cost. And further improve reliability by selecting redundant servers.

[0091] Delay Analysis: 1) For computing delay, the common method of calculating computing power is to calculate based on the CPU frequency of each edge server. The present invention adds the CPU load of the edge server to calculate. Because in practice, even if the edge servers maintain the same CPU frequency, the computing power will vary greatly.

[0092] The following is a brief introduction to the method of calculating the task computing time and communication time:

[0093] 1) For calculation delay, Represents edge server The CPU load, Expressed as a percentage; Indicates the size of the input data, Expressed in bits; the number of CPU cycles required to decode one bit of a frame is defined as the decoding complexity ; The number of CPU cycles required to detect an object in a frame is defined as the detection complexity ; The number of CPU cycles required for decoding is recorded as , ; The number of CPU cycles required for detection is recorded as ;The number of CPU cycles required to process a frame is . Edge Server The CPU frequency is recorded as .

[0094] In addition, consider the CPU load . CPU Load It is calculated as 1 minus the ratio of CPU idle time to total CPU time, indicating the proportion of CPU resources used.

[0095] Therefore, as CPU idle time increases, CPU load decreases and CPU resources increase. is a The computing time function is used to measure the computing time trend. The CPU load decreases, the CPU resources increase, and the computing time is shorter, that is, the edge computing part calculates faster.

[0096] Video subframe On edge servers The computation time on is: .

[0097] 2) For communication latency, the data center part to the edge server The data transfer rate (in bit / s) can be obtained by Obtain, among which It is an edge server RSSI, It is a function of the data transmission rate with respect to RSSI, that is, the calculation formula of the data transmission rate.

[0098] Data center to edge server The communication time is Communication time From data center to edge server Uninstall time and edge server Download time to the data center The size of the test result data is relatively small compared to the input data, so the download time is set to a fixed constant value based on the round-trip time (RTT). .

[0099] Therefore, the video subframe On edge servers The processing delay on can be calculated as:

[0100] .

[0101] 3) Payment analysis: Payment can be measured by the CPU resources contributed by the edge server, that is, it is determined by the CPU load and the amount of data processed. The decrease in CPU load and the increase in the amount of data processed will lead to a linear increase in payment. On edge servers Payment It can be calculated as .in Express payment in units (cents / GHz / KB).

[0102] 4) Cost analysis: The cost is calculated by weighting the payment and delay. Video subframe On edge servers Cost It can be calculated as:

[0103] .

[0104] in, and are the weighting coefficients of payment and delay, respectively.

[0105] Total cost of the current frame It can be calculated as:

[0106] .

[0107] 5) Reliability Guarantee: Reliability is measured by task completion rate, which reflects the impact of user mobility and resource fluctuations. If a subtask is offloaded to the edge server and the data center receives the execution result from the edge server before the deadline, the subtask is considered to be successfully completed. Due to the heterogeneity of mobility and CPU resources, the completion probability of each edge server for a task may change at different times. To this end, we use historical performance data over a period of time to update the reliability of each device.

[0108] For estimating subtasks Complete reliability. Edge Server The reliability of , it is calculated as the ratio of the number of subtasks successfully executed within each heartbeat time period (for example, 2 seconds) to the total number of subtasks.

[0109] It can be calculated as:

[0110]

[0111] If the subtask The reliability is below the mission reliability threshold (set up ), then select redundant edge servers one by one until the reliability requirement is met. Then the edge server set excludes the selected edge servers and is sorted in ascending order of cost to select the subtask Whenever deciding whether to select an edge server, the edge server reliability must be greater than the minimum threshold (set up ), to avoid selecting too many redundant edge servers. At the same time, the deadline requirement must be met. And to avoid queue delay, each edge server cannot be selected twice in a set of video subframes.

[0112] 6) Offloading control: Offloading control aims to coordinate the detection process between the data center part and the edge server. The key idea is to minimize the cost of each subtask in a greedy manner according to the channel quality, available CPU resources and task completion reliability, thereby minimizing the total cost. Then, the optimal set of edge servers for each subframe is determined.

[0113] Specifically, first initialize the optimal edge server set for each frame , the optimal edge server set for each video subframe , the set of selected edge servers for each frame . Each frame is then divided into Video subframes. Start selecting video subframes The best edge server To avoid queue delay, each edge server cannot be selected twice in a set of video subframes. For each video subframe, the edge server with the lowest cost in the edge server set is selected first, and the selected edge server must meet the deadline requirement. If no edge server meets the deadline requirement, the edge server with the smallest delay will be directly selected. If the subtask The reliability is below the reliability threshold (set up ), then select redundant edge servers one by one until the reliability requirement is met. Then the edge server set excludes the selected edge servers and is sorted in ascending order of cost to select the subtask Whenever deciding whether to select an edge server, the reliability must be greater than the minimum threshold (set up ), to avoid selecting too many redundant edge servers. At the same time, the deadline requirements must be met.

[0114] The main work of the present invention is how to discover available devices and how to offload tasks. Among them, the access point will periodically send beacons embedded with the required services to discover available devices as edge servers. The detection model is run locally for initial target detection. Then the targets across any boundaries are extracted. In order to improve the detection accuracy and shorten the decoding time, each frame is cropped into multiple slices for fine-grained detection on the selected server. The effective redundant server mechanism ensures the reliability of task completion. In addition, the CPU load of the server is used to estimate its computing power and the computing power is used to estimate the resource payment. Finally, the local and remote detected targets are integrated to obtain the final target detection results, which provides a basis for the development of low-cost and reliable systems. Through the proposed lightweight task splitting, redundant edge selection and fast service discovery scheme, the present invention can offload video frames to edge mobile devices in a lightweight and opportunistic manner, and use them to control traffic lights in real time.

[0115] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A traffic light control method based on environmental edge computing, characterized in that: The steps include: Step S1, the data center part periodically broadcasts a WiFi beacon embedded with a request service through an access point, and discovers a mobile device that can serve as an edge server through a service manager; Step S2: The camera starts to collect video frames, and each frame is transmitted through two paths; one is used for local detection by the local detector in the data center part, and the other is used for remote detection in the edge computing part; Step S3: The data center runs the YOLOv5s network detection model on the local detector, performs analysis of the video frame in the local area with limited resources, and performs initial target detection, i.e., target positioning and classification; divides the video frame into boundary lines, extracts the targets crossing the boundary lines according to the coordinate position information of the detected targets, and sends the index of the frame where the target is located and the target detection result to the integration manager; Step S4: the cropping manager of the data center part crops each video frame into several main slices according to the boundary lines; Each main slice is then padded with surrounding padding slices to create a new video subframe; this is then encoded into a byte stream that can be used for transmission; Step S5: The offload manager of the data center part allocates a corresponding edge server to each video subframe through a redundant server selection mechanism according to the CPU load and RSSI information of each edge server transmitted by the service manager; and assembles the IP, port and encoded video subframe corresponding to the edge server into transmission data and stores it in a sending queue, waiting to be sent to the selected edge server; Step S6: The edge server of the edge computing part receives the transmission data sent by the data center part, decodes the transmission data in the receiving queue, performs target detection through the Nanodet-Plus target detection model of the remote detector, and sends the current video subframe index and target detection result obtained by the target detection to the data center part; Step S7: The integration manager of the data center integrates the target detection result obtained by local detection and the target detection result obtained by remote detection of the edge server to obtain a final target detection result, and controls the traffic light according to the final target detection result; In step S5, the offload manager of the data center part is used to accelerate the detection process to complete the real-time task; the redundant server selection mechanism is implemented in the following way: for each subtask / video subframe, the cost of each edge server is estimated according to the end-to-end delay and payment; then, the edge server with the lowest cost and meeting the deadline requirement is selected; if no edge server meets the deadline requirement, the edge server with the smallest end-to-end delay is selected to ensure that the current subtask is not missed; After selecting the first optimal edge server for each subtask, decide whether to continue selecting more edge servers based on the reliability of the current subtask; Whenever a video frame is generated, it is divided into video subframes, and the video subframe set is ;have The edge servers that can provide services are connected to the data center part, and their collection is ;use Represents a video subframe Is it assigned to the edge server? ; Video subframe On edge servers The computation time on is: in, The number of CPU cycles required for decoding; The number of CPU cycles required for detection; Represents edge server CPU load; is a The calculation time function of For edge servers CPU frequency; Data center to edge server The communication time is: in, Indicates the size of the input data; From data center to edge server The data transfer rate, Represents edge server The time to download the test result data to the data center; Video subframe On edge servers The processing delay on is calculated as: 。 2. The traffic light control method based on environmental edge computing according to claim 1, characterized in that: In step S1, the data center embeds service information in the beacon to allow the data center to discover mobile devices that can provide the requested service; nearby mobile devices perform service queries by parsing the beacon information; if the mobile device can provide the service requested by the data center, it will actively establish a WiFi connection with the data center and become an edge server.

3. The traffic light control method based on environmental edge computing according to claim 2 is characterized in that: The data center part has a Bloom filter and provides a list of requested services; each edge device has a Bloom filter and provides a list of available services; in the service map of the data center part, the bit array Initially, it is set to an array of all 0s; then the data center part hashes the name of each requested service in the list of requested services, maps it to the corresponding position through multiple hash functions, and converts the bit array Set the bits corresponding to these positions in the bit array to 1; then Embedded in beacons, broadcast beacons periodically; In the service query of the edge computing part, each edge device parses the received beacon to obtain the bit array , and query the list of services it can provide; The edge device performs a hash mapping on the name of each service it can provide, and the mapping result is the same as the received bit array. Compare, the bit with mapping result of 1 is in the bit array Only when the corresponding bits in the data center are all 1, the edge device has the service requested by the data center; if the edge device has at least one service that meets the requirements, that is, at least one service that the edge device can provide matches the service requested by the data center, then the edge device is connected to the wireless network of the data center and becomes an edge server.

4. The traffic light control method based on environmental edge computing according to claim 3 is characterized in that: The edge server sends a network connection request to the data center. After the connection is successful, the edge server registers the device status and service information with the data center. The data center initiates a service request to the edge computing part according to the edge offloading needs. The edge computing part processes the request and returns the result.

5. The traffic light control method based on environmental edge computing according to claim 1, characterized in that: Before the data center cuts each video frame into several main slices, the padding size is determined in advance; the target category to be detected is recorded as , including people, bicycles, cars, motorcycles, buses, and trucks; for each category , the minimum width and minimum height that the local detector can detect are recorded as ,in Indicates category The minimum width of Indicates category The minimum height of Represents the minimum width and minimum height of all categories. Indicates the minimum width of all categories. Represents the minimum height of all categories; the width threshold and height threshold of small objects are determined by selecting the maximum value of the product of the minimum width and the minimum height among all categories, denoted as ,in Indicates the minimum detectable target width, represents the minimum detectable object height; the padding width is then determined by halving the width and height thresholds for small objects and fill height ,Right now ; The data center evenly divides the complete video frame into multiple main slices according to the available edge servers. The width and height of each main slice are recorded as and ,Right now ;in Represents the index of each main slice, represents the number of edge servers, and Represents the width and height of a complete video frame; in addition, each main slice will be resized accordingly after padding to form a video subframe.

6. The traffic light control method based on environmental edge computing according to claim 5, characterized in that: The filling rule for each main piece is as follows: each main piece contains two edge lines of width and two edge lines of height. If its edge line does not overlap with the edge line of the complete frame, the edge line is filled; if a main piece is filled once in the width direction, the adjusted width is: Similarly, if the height is filled once, the adjusted height is: .

7. The traffic light control method based on environmental edge computing according to claim 1, characterized in that: Video subframe On edge servers Payment Calculated as: in, Indicates unit payment; On edge servers Cost Calculated as: in, and are the weighting coefficients of payment and delay, respectively; Total cost of the current frame It can be calculated as: 。 8. The traffic light control method based on environmental edge computing according to claim 7, characterized in that: After selecting the first optimal edge server for each subtask, the decision on whether to continue selecting more edge servers is made based on the reliability of the current subtask in the following way: If the reliability of a subtask is lower than the task reliability threshold, redundant edge servers are selected one by one until the reliability requirement is met; then the edge server set excludes the selected edge servers and is sorted in ascending order of cost to select the redundant edge servers of the subtask; whenever deciding whether to select an edge server, the edge server reliability must be greater than the minimum threshold to avoid selecting too many redundant edge servers; at the same time, the deadline requirement must be met; and in order to avoid queue delays, each edge server cannot be selected twice in a set of video subframes.

Images