Vehicle control method and device, server and storage medium

Abstract

The application provides a vehicle control method and device, a server and a storage medium. The method is applied to an edge server deployed at any driving node of a driving route, and includes the following steps: receiving a vehicle surrounding dynamic video sent by at least one vehicle terminal in the driving node, wherein the vehicle surrounding dynamic video is collected by a camera device on the at least one vehicle terminal; receiving a road static video sent by a roadside device corresponding to the driving node; judging whether a safety abnormal event occurs according to the vehicle surrounding dynamic video and the road static video; if it is judged that the safety abnormal event occurs, obtaining a judgment result of the safety abnormal event, and broadcasting the judgment result of the safety abnormal event to all passing vehicles in a driving node coverage range of the edge server, so that the decision accuracy of the vehicle when encountering an accident problem is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of Internet of Vehicles, and in particular to a vehicle control method and device, a server and a storage medium. BACKGROUND

[0002] The Internet of Vehicles video technology mainly relies on a camera on a vehicle to identify a scene in a road, and according to a video identification result, the vehicle takes measures to respond to various situations on the road.

[0003] In the prior art, the camera on the vehicle acquires a safety accident scene in the road, and after resource allocation, the semantic of the video data collected by the camera is understood and analyzed, and according to the processing result, intelligent decision-making of the safety class and the information class and vehicle control are completed.

[0004] However, in the prior art, the video acquisition only relies on the vehicle itself to acquire the scene in the horizontal dimension, and the video content is limited, so that the vehicle is not accurate enough when making decisions in the event of an accident. SUMMARY

[0005] The present application provides a vehicle control method, device, server and storage medium to solve the problem that the vehicle is not accurate enough when making decisions in the event of an accident in the prior art.

[0006] In a first aspect, the present application provides a vehicle control method applied to an edge server deployed at any driving node of a driving route, comprising:

[0007] receiving a vehicle surrounding dynamic video sent by at least one vehicle terminal in the driving node, wherein the vehicle surrounding dynamic video is collected by a camera device on the at least one vehicle terminal;

[0008] receiving a road static video sent by a roadside device corresponding to the driving node;

[0009] judging whether a safety abnormal event occurs according to the vehicle surrounding dynamic video and the road static video;

[0010] if it is judged that the safety abnormal event occurs, obtaining a judgment result of the safety abnormal event, and broadcasting the judgment result of the safety abnormal event to all passing vehicles in a driving node coverage range to which the edge server belongs.

[0011] In an embodiment of the present application, the judging whether a safety abnormal event occurs according to the vehicle surrounding dynamic video and the road static video comprises: identifying a relatively static scene existing in the road static video by a target detection algorithm; obtaining basic road information according to the vehicle surrounding dynamic video; and judging whether a safety abnormal event occurs according to the relatively static scene and the basic road information.

[0012] In one embodiment of this application, determining whether a safety anomaly has occurred based on the relatively static scene and the basic road information includes: inputting the relatively static scene and the basic road information into a trained machine learning model to output a determination result of the safety anomaly.

[0013] In one embodiment of this application, after receiving the dynamic video of the vehicle's surroundings sent by at least one vehicle within the driving node, the method further includes: receiving the recognition result of a relatively static scene sent by the roadside device corresponding to the driving node; wherein the recognition result of the relatively static scene is obtained by the roadside device after matching the collected road static video with a preset data model; determining whether a safety anomaly event has occurred based on the recognition result of the relatively static scene and the dynamic video of the vehicle's surroundings; if a safety anomaly event is determined to have occurred, obtaining the determination result of the safety anomaly event and broadcasting the determination result of the safety anomaly event to all passing vehicles within the coverage area of ​​the driving node to which the edge server belongs.

[0014] In one embodiment of this application, it further includes:

[0015] The relationship between video content understanding accuracy and latency on the edge server and vehicle end is optimized using the following formula to achieve the optimal configuration of video content understanding accuracy and latency:

[0016]

[0017]

[0018] C2:B c B r ≥B min

[0019] C3:R n,c ,R n,r ≤R m

[0020] C4:T′ n (B c B r )≤T n

[0021] C5:E n ≤E max

[0022] C6:θ c ,θ v ≥θ min

[0023] In the formula, P1 represents the relationship model between video content understanding accuracy and latency; B nB denotes bandwidth; N denotes total time, n denotes each time; θ n B denotes the importance weight of the roadside abnormal situation; Target n P n B denotes the number of correctly detected abnormal scene targets, mAp denotes the mean value of the average precision, wherein B denotes the probability of correctly detecting targets at the vehicle end; B denotes the probability of correctly detecting targets at the roadside device; The q() function is used to represent the relationship model between video quality (QP value) and video code rate, wherein q(R n,c ) denotes the relationship model at the vehicle side, q(R n,r ) denotes the relationship model at the roadside; ρ denotes a compensation factor;

[0024] C1-C6 denote constraint conditions of optimal configuration;

[0025] C1 denotes that the bandwidth resources of the vehicle end and the roadside device cannot be greater than the total bandwidth resources allocated by the vehicle-road system, wherein B c denotes the uplink bandwidth of each roadside device; B r denotes the uplink transmission channel bandwidth of the vehicle end; B max denotes the total bandwidth resources allocated by the vehicle-road system;

[0026] C2 denotes that the bandwidth resources of the vehicle end and the roadside device are at least greater than the transmission bandwidth threshold of the vehicle-road system, wherein B min denotes the transmission bandwidth threshold of the vehicle-road system;

[0027] C3 denotes that the transmission rates of the vehicle end and the roadside device cannot be greater than the theoretical maximum data rate allowed by the Shannon formula; wherein R n,c denotes the transmission rate of the vehicle end; R n,r denotes the transmission rate of the roadside device; R m denotes the theoretical maximum data rate allowed by the Shannon formula;

[0028] C4 denotes that the calculation delay of the transmission and calculation process cannot be greater than the total delay requirement for completing the target detection task; wherein T′ n (B c ,B r ) denotes the calculation delay of the transmission and calculation process; T n denotes the total delay requirement for completing the target detection task;

[0029] C5 denotes that the total energy consumption of the video content understanding task cannot be greater than the energy consumption limit in the task offloading process; wherein E n denotes the total energy consumption of the video content understanding task; E max denotes the energy consumption limit in the task offloading process;

[0030] C6 represents that the vehicle-side target detection accuracy threshold and the edge server target detection accuracy threshold are greater than the video target detection accuracy minimum threshold; wherein θ c represents the vehicle-side target detection accuracy threshold; θ v represents the edge server target detection accuracy threshold; θ min represents the video target detection accuracy minimum threshold.

[0031] In a second aspect, the present application provides a vehicle control device, applied to an edge server deployed at any driving node of a driving route, comprising:

[0032] A first receiving module, configured to receive a vehicle surrounding dynamic video sent by at least one vehicle side in the driving node, wherein the vehicle surrounding dynamic video is collected by a camera device on the at least one vehicle side;

[0033] A second receiving module, configured to receive a road static video sent by a roadside device corresponding to the driving node;

[0034] A first judging module, configured to judge whether a safety abnormal event occurs according to the vehicle surrounding dynamic video and the road static video;

[0035] A first broadcasting module, configured to, if it is judged that the safety abnormal event occurs, acquire a judgment result of the safety abnormal event, and broadcast the judgment result of the safety abnormal event to all passing vehicles in a driving node coverage range to which the edge server belongs.

[0036] In an embodiment of the present application, the first judging module is specifically configured to identify a relatively static scene existing in the road static video through a target detection algorithm; acquire basic road information according to the vehicle surrounding dynamic video; and judge whether a safety abnormal event occurs according to the relatively static scene and the basic road information.

[0037] In an embodiment of the present application, the first judging module is further specifically configured to input the relatively static scene and the basic road information into a trained machine learning model to output a judgment result of the safety abnormal event.

[0038] In a third aspect, the present application provides an edge server, comprising:

[0039] at least one processor and a memory;

[0040] The memory stores computer execution instructions;

[0041] The at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the vehicle control method of the first aspect.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the processor executes the computer-executable instructions, the vehicle control method of the first aspect is implemented.

[0043] The vehicle control method, device, server and storage medium provided by the present application enrich the video content by combining the dynamic video around the vehicle and the static video of the road, and can improve the accuracy of the decision-making of the vehicle in the event of an accident. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0045] Figure 1 The application scenario of the vehicle control method provided by the present application is shown in the figure.

[0046] Figure 2 The flowchart of the vehicle control method provided by an embodiment of the present application is shown in the figure.

[0047] Figure 3 The structure diagram of the vehicle control device provided by an embodiment of the present application is shown in the figure.

[0048] Figure 4 The hardware structure diagram of the edge server provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0050] The existing vehicle networking video technology mainly relies on the camera on the vehicle to collect and identify the scene in the road, and according to the identification result, the vehicle takes measures to deal with the situation on the road. After the camera collects video information, the content in the video is understood and analyzed, and the vehicle is controlled to make a decision action according to the result obtained. However, in the prior art, the acquisition of video only relies on the vehicle itself to obtain the horizontal scene dimension, and the video content obtained is limited, and when the vehicle encounters an accident problem that needs to be decided, the reaction is not timely enough.

[0051] To solve the above technical problems, the embodiment of the application proposes the following technical idea: on the basis of acquiring the vehicle peripheral dynamic video collected by the vehicle end, the road static video collected by the roadside is considered to be added; the vehicle peripheral dynamic video and the road static video are combined to comprehensively judge whether to send a safety abnormal event, thereby expanding the accident scene video acquisition range and improving the identification accuracy of the safety abnormal event.

[0052] Figure 1 is the application scenario of the vehicle control method provided by the embodiment of the application. As shown in Figure 1 , the application scenario of the vehicle control method includes a vehicle end 101, a roadside device 102, a road scene 103, and an edge server 104.

[0053] Among them, the vehicle end 101 includes a vehicle body 1011 and a camera device 1012 arranged on the vehicle body. Among them, the camera device 1012.

[0054] Among them, the roadside device 102 includes a roadside camera 1021 and a roadside unit RSU (Road Side Unit, RSU for short) 1022.

[0055] Among them, the road scene 103 has a set driving route, and the driving route includes a plurality of driving nodes, and each driving node is deployed with a roadside device 102 and an edge server 104.

[0056] Among them, the edge server 104 can be a server or a cluster composed of multiple servers.

[0057] Here, the vehicle end 101 collects the vehicle peripheral dynamic video through the camera device on the vehicle, and sends the vehicle peripheral dynamic video to the edge server 104. The roadside device 102 identifies the road scene 103 in the road, acquires the road static video, and sends the road static video to the edge server 104. The edge server 104 judges whether a safety abnormal event occurs according to the vehicle peripheral dynamic video and the road static video, and if a safety abnormal event occurs, broadcasts the judgment result of the safety abnormal event to all vehicles in the driving node coverage range of the edge server 104.

[0058] The following is described in detail with detailed embodiments.

[0059] Embodiment One

[0060] Figure 2 The vehicle control method flowchart provided for an embodiment of the application, the execution subject of the embodiment can be Figure 1 The edge server 104 deployed at any driving node of the driving route in the embodiment, the embodiment is not particularly limited here. As Figure 2 The method comprises:

[0061] S201: receiving a vehicle surrounding dynamic video sent by at least one vehicle terminal in the driving node, wherein the vehicle surrounding dynamic video is collected by a camera device on the at least one vehicle terminal.

[0062] In the embodiment, there can be multiple vehicle terminals in the driving node; wherein the at least one vehicle terminal in the driving node can be any one of the multiple vehicle terminals, or at least two or more of the multiple vehicle terminals.

[0063] In the embodiment, the vehicle surrounding dynamic video is a dynamic video of an object in front of the vehicle relative to the vehicle during the driving process of the vehicle, which is captured by a camera device on the vehicle.

[0064] Specifically, the vehicle surrounding dynamic video can be sent by the communication module on the vehicle terminal through wireless communication.

[0065] S202: receiving a road static video sent by a roadside device corresponding to the driving node.

[0066] In the embodiment, the road static video is a video of a static collision accident or a static obstacle in the driving node range of the roadside device obtained by a roadside camera of the roadside device.

[0067] In the embodiment, the roadside camera of the roadside device collects the road scene of the driving node where the roadside device is located to obtain the road static video, and sends the road static video to the RSU.

[0068] Specifically, the road static video sent by the RSU of the roadside device can be received through wireless communication.

[0069] S203: judging whether a safety abnormal event occurs according to the vehicle surrounding dynamic video and the road static video.

[0070] Specifically, step S203 specifically comprises:

[0071] S2031: identifying a relatively static scene existing in the road static video by a target detection algorithm.

[0072] In this embodiment, the target detection algorithm includes but is not limited to R-CNN series algorithm, YOLO series algorithm, SSD series algorithm, RetinaNet algorithm, etc.

[0073] In this embodiment, the relatively static scene includes a static collision accident scene and a static obstacle scene.

[0074] S2032: Obtain basic road information according to the vehicle surrounding dynamic video.

[0075] Specifically, the road scene appearing in the video is information extracted according to the content of the vehicle surrounding dynamic video to obtain the basic road information.

[0076] Among them, the basic road information can include information such as whether the road is smooth, the road congestion state, and the traffic light state on the road.

[0077] S2033: Determine whether a safety abnormal event occurs according to the relatively static scene and the basic road information.

[0078] Specifically, step S2033 specifically includes: inputting the relatively static scene and the basic road information into the trained machine learning model to output the determination result of the safety abnormal event.

[0079] In this embodiment, the safety abnormal event includes but is not limited to vehicle collision accidents in the road, unknown objects in the road center, road prohibitions caused by road construction, traffic control, etc.

[0080] In this embodiment, the trained machine learning model is obtained by iteratively training the machine learning model using a large number of historical vehicle surrounding dynamic videos and road static videos.

[0081] S204: If it is determined that a safety abnormal event occurs, obtain the determination result of the safety abnormal event, and broadcast the determination result of the safety abnormal event to all passing vehicles in the driving node coverage range to which the edge server belongs.

[0082] Specifically, the determination result of the safety abnormal event can be broadcast to all passing vehicles in the driving node coverage range to which the edge server belongs through wireless communication.

[0083] In this embodiment, after all the passing vehicles in the driving node coverage range receive the determination result of the safety abnormal event, the accident problem is decided according to the determination result of the safety abnormal event to control the vehicle driving.

[0084] From the above embodiment, the edge server first acquires the vehicle peripheral dynamic video through the vehicle end, then acquires the road static video through the roadside device, and then judges whether a safety abnormal event occurs according to the vehicle peripheral dynamic video acquired by the vehicle and the road static video, and broadcasts the determination result to the vehicles in the coverage range of the driving node. Since the vehicle peripheral dynamic video and the road static video are combined, the video content is enriched, and the accuracy of the vehicle accident problem decision can be improved.

[0085] Embodiment Two

[0086] In an embodiment of the present application, in Figure 2 Based on the embodiment, the steps of the present example are described as follows:

[0087] S301: receiving vehicle peripheral dynamic video sent by at least one vehicle end in a driving node, wherein the vehicle peripheral dynamic video is collected by a camera device on the at least one vehicle end.

[0088] In the present embodiment, the description of step S301 is consistent with step S201, and the specific content is referred to the description of step S201, which will not be repeated here.

[0089] S302: receiving an identification result of a relatively static scene sent by a roadside device corresponding to the driving node; wherein the identification result of the relatively static scene is obtained by matching a road static video collected by the roadside device with a preset data model.

[0090] Specifically, the roadside camera in the roadside device collects the road static video, and sends the road static video to the RSU. The RSU matches the road static video with the preset data model to obtain the identification result of the relatively static scene. The RSU transmits the identification result of the relatively static scene to the edge server through wireless transmission.

[0091] In the present embodiment, the preset data model includes a road traffic accident data model.

[0092] S303: judging whether a safety abnormal event occurs according to the identification result of the relatively static scene and the vehicle peripheral dynamic video.

[0093] In the present embodiment, the description of step S303 is consistent with step S2033, and the specific content is referred to the description of step S2033, which will not be repeated here.

[0094] S304: if it is determined that a safety abnormal event occurs, obtaining a determination result of the safety abnormal event, and broadcasting the determination result of the safety abnormal event to all passing vehicles in the coverage range of the driving node to which the edge server belongs.

[0095] In the embodiment, the description of step S304 is consistent with step S204, and the specific content can be referred to the description of step S204, which will not be repeated here.

[0096] From the above embodiment, in the embodiment, the roadside device obtains the recognition result of the relatively static scene after matching the collected road static video with the preset data model, the edge server receives the recognition result of the relatively static scene sent by the roadside device, and then obtains the dynamic video around the vehicle through the vehicle end. According to the recognition result of the relatively static scene and the dynamic video around the vehicle, it is judged whether a safety abnormal event occurs, and the determination result is broadcast to the vehicles in the coverage range of the running node. Not only can the edge server broadcast the result of judging whether a safety abnormal event occurs to the vehicles in the coverage range of the running node faster, but also increases the speed of decision-making when the vehicle encounters an accident. Moreover, since the roadside device matches the obtained road static video with the preset data model and only sends the recognition result of the relatively static scene to the edge server, the amount of data transmission is reduced, and the processing efficiency is improved.

[0097] Embodiment three

[0098] In an embodiment of the present application, based on the above-mentioned embodiments, the present example further includes a process of optimizing the resources of the edge server and the vehicle end, which is described in detail as follows:

[0099] According to the following formula, the relationship between the video content understanding accuracy and the delay of the edge server and the vehicle end is optimized, and the optimal configuration of the video content understanding accuracy and the delay is realized:

[0100]

[0101]

[0102] C2: B c ,B r ≥ B min

[0103] C3: R n,c ,R n,r ≤ R m

[0104] C4: T' n (B c ,B r ) ≤ T n

[0105] C5: E n ≤ E max

[0106] C6: θ c , θ v ≥ θ min

[0107] P1 represents the relationship model between video content understanding accuracy and latency; B n represents bandwidth; N represents total time, n represents each time; θ n represents the importance weight of the roadside abnormal situation; Target n P n represents the number of correctly detected abnormal scene targets, mAp represents the mean value of average precision, wherein represents the probability of correctly detecting targets at the vehicle end; represents the probability of correctly detecting targets at the roadside device; q() function is used to represent the relationship model between video quality (QP value) and video code rate, wherein q(R n,c ) represents the relationship model at the vehicle side, q(R n,r ) represents the relationship model at the roadside; p represents a compensation factor;

[0108] C1-C6 represent constraint conditions of optimal configuration;

[0109] C1 represents that the bandwidth resources of the vehicle and the roadside cannot be greater than the total bandwidth resources allocated in advance, wherein B c represents the uplink bandwidth of each roadside camera; B r represents the uplink transmission channel bandwidth of the vehicle; B max represents the total bandwidth resources allocated by the vehicle-road system.

[0110] C2 represents that the bandwidth resources of the vehicle and the roadside are at least greater than a lower limit value, wherein B min represents the transmission bandwidth threshold under the vehicle-road cooperative scenario.

[0111] C3 represents that the transmission rates of the vehicle and the roadside cannot be greater than the theoretical maximum data rate allowed by the Shannon formula; wherein R n,c represents the transmission rate at the vehicle side; R n,r represents the transmission rate at the roadside; R m represents the maximum data rate allowed by the Shannon formula.

[0112] C4 represents that the calculation latency of the transmission and calculation process cannot be greater than the total latency of the task; wherein T′ n (B c ,B r ) represents the calculation latency of the transmission and calculation process; T n represents the allowed latency requirement for completing the target detection task.

[0113] C5 represents that the total energy consumption of the video content understanding task cannot be greater than the energy consumption limit in the task offloading process; wherein E n represents the total energy consumption of the video content understanding task; E maxrepresents the energy consumption limit in the task offloading process.

[0114] C6 represents that the vehicle-side and edge-side target detection accuracy is greater than a minimum threshold value; wherein θ c represents a vehicle-side target detection accuracy threshold value; θ v represents an edge-side target detection accuracy threshold value; θ min represents a video target detection accuracy threshold value.

[0115] wherein the q() function is used to represent a relationship model of video quality (QP value) and video code rate, wherein q(R n,c ) represents a relationship model of the vehicle side, q(R n,r ) represents a relationship model of the road side, and the descriptions are as follows:

[0116] In order to achieve better target detection accuracy, the original video is compressed at a larger code rate at the vehicle end, so as to obtain higher video quality, and the relationship between video quality (QP value) and target detection accuracy P m can be recorded as:

[0117] P m (QP)=α·QP β +γ

[0118] In the formula, P m represents target detection accuracy; QP represents video quality; and α, β, and γ are model parameters.

[0119] It can be seen that the size of the video code rate directly determines the video quality and the final video file size, the video quality QP determines the video compression rate, and the video code rate is related to the compression rate. Therefore, the relationship model of video quality (QP value) and video code rate can be represented as:

[0120] QP=a·exp(b·R t )

[0121] In the formula, a and b are model parameters, which can be respectively taken as 46.27 and -7.086e-5, exp() represents an exponential function with e as the base, R t represents transmission rate, when the above formula calculates the video quality of the vehicle end, R t can be represented by R n,c ; and when the above formula calculates the video quality of the road side device, R t can be represented by R n,r .

[0122] According to the relationship model formula of video quality and video code rate, the calculation formula of transmission rate R t can be derived:

[0123]

[0124] where a and b are model parameters, which can be 46.27 and -7.086e-5, respectively, R t denotes the transmission rate, and QP denotes the video quality.

[0125] Specifically, the transmission rate R n,c The calculation formula is as follows:

[0126] where, the transmission rate R n,c is a time function of t.

[0127]

[0128] where B c denotes the uplink bandwidth of the roadside camera in the roadside device, P c denotes the transmission power, h c (t) denotes the channel gain, t denotes time, σ 2 denotes the Gaussian white noise power, and lb() denotes the log base 2 logarithm.

[0129] Specifically, the transmission rate R n,r The calculation formula is as follows:

[0130] where, the transmission rate R n,r is a time function of t.

[0131]

[0132] where B r denotes the uplink transmission channel bandwidth of the vehicle end, P v denotes the average transmission power of the vehicle end, h denotes the uplink channel fading factor, σ 2 denotes the Gaussian white noise power, lb() denotes the log base 2 logarithm, d -δ denotes the path loss between the vehicle end and the edge node, and δ denotes the loss factor, where the calculation formula of d is as follows:

[0133]

[0134] where l denotes the vertical distance corresponding to the vehicle and the edge node, r denotes the radius distance of the edge node coverage range, v n denotes that the connection rate of the vehicle end and the edge node satisfies the Poisson distribution.

[0135] In order to simplify the system problem, the average transmission rate is used instead of the time-varying transmission rate,

[0136]

[0137]

[0138] Among them, t n t represents the time change at the vehicle end. i This indicates the time-related changes of roadside equipment.

[0139] Specifically, in C4, T′ n (B c The delay T represents the latency of the transmission and computation processes at the vehicle end. n,car The calculation formula is as follows:

[0140]

[0141] Among them, D n,c D represents the amount of video data transmitted. n,m D represents the amount of decision-making task in the model. n,s This indicates the amount of data from the vehicle's onboard sensors. The average upload rate of the vehicle is N, where N represents the total time and n represents each moment.

[0142] Specifically, the time delay T of the vehicle-side calculation process n,local The calculation formula is as follows:

[0143]

[0144] Among them, D′ n,c D′ represents the amount of video data transmitted on the edge server side. n,m D′ represents the model decision-making task on the edge server side. n,s This represents the amount of vehicle-mounted sensor data on the edge server side. This represents the average local upload rate of the vehicle, where N represents the total time and n represents each moment.

[0145] Specifically, in C4, T′ n (B r The delay T represents the latency of the transmission and computation processes between roadside equipment and edge servers. n,road The latency T during the transmission process with the edge server n,mec The calculation formula is as follows:

[0146]

[0147]

[0148] Among them, D n,r This indicates the amount of video data from roadside equipment. D' represents the average upload rate of the roadside device n D represents the amount of video data on the edge server side.

[0149] Specifically, the computing delay T of the edge server side n,m The calculation formula is as follows:

[0150]

[0151] wherein D n D represents the amount of data on the edge server side, C represents the processing frequency on the edge server side.

[0152] Specifically, the total energy consumption E of the video content understanding task in C5 n The calculation formula is as follows:

[0153] E n = e n,c + e n,t

[0154] wherein e n,c represents the computing energy consumption, e n,t represents the transmission energy consumption.

[0155] Specifically, the calculation formula of the computing energy consumption e n,c is as follows:

[0156]

[0157] wherein ζ represents the effective capacitance coefficient of the chip, D n,m is the data volume of the video understanding task, U n,m represents the computing period required for the unit bit task volume, represents the available computing resources.

[0158] Specifically, the calculation formula of the transmission energy consumption e n,t is as follows:

[0159]

[0160] wherein P c represents the transmission power, P v represents the average transmission power on the vehicle side, represents the average transmission rate on the vehicle side, represents the average transmission rate of the roadside device.

[0161] From the above embodiments, by establishing the relationship between the video content understanding accuracy and the time delay of the edge server and the vehicle side, the vehicle side compresses the dynamic video around the vehicle at a larger code rate to obtain a higher quality dynamic video around the vehicle, and unloads the dynamic video around the vehicle obtained by the vehicle side to the edge server for operation on the edge server side, thereby solving the problem of insufficient computing power of the vehicle side, reducing the time delay of the entire system, and achieving the effect of reducing the time delay while ensuring the video content understanding accuracy.

[0162] Embodiment Four

[0163] Figure 3 A structural schematic diagram of a vehicle control device provided by an embodiment of the present application is shown in FIG. 4, which comprises a first receiving module 401, a second receiving module 402, a first judging module 403 and a first obtaining module 404. Figure 3

[0164] The first receiving module 401 is configured to receive a dynamic video around a vehicle sent by at least one vehicle side in a driving node, wherein the dynamic video around the vehicle is collected by a camera device on the at least one vehicle side.

[0165] The second receiving module 402 is configured to receive a road static video sent by a roadside device corresponding to the driving node.

[0166] The first judging module 403 is configured to judge whether a safety abnormal event occurs according to the dynamic video around the vehicle and the road static video.

[0167] The first broadcasting module 404 is configured to, if it is determined that the safety abnormal event occurs, obtain a determination result of the safety abnormal event, and broadcast the determination result of the safety abnormal event to all passing vehicles in a coverage range of a driving node to which the edge server belongs.

[0168] The device provided by the embodiment can be used to execute the technical solutions of the method embodiments, and has similar implementation principles and technical effects, which will not be described here again.

[0169] In an embodiment of the present application, the first judging module 403 is specifically configured to identify a relatively static scene existing in the road static video by a target detection algorithm; obtain basic road information according to the dynamic video around the vehicle; and judge whether the safety abnormal event occurs according to the relatively static scene and the basic road information.

[0170] In an embodiment of the present application, the first judging module 403 is specifically configured to input the relatively static scene and the basic road information into a trained machine learning model to output the determination result of the safety abnormal event.

[0171] ​In an embodiment of the present application, the device 40 further comprises:

[0172] The third receiving module 405 is configured to receive the identification result of the relative static scene sent by the roadside device corresponding to the travel node, wherein the identification result of the relative static scene is obtained by matching the collected road static video with a preset data model by the roadside device.

[0173] The second judging module 406 is configured to judge whether a safety abnormal event occurs according to the identification result of the relative static scene and the dynamic video around the vehicle.

[0174] The second broadcasting module 407 is configured to, if it is determined that the safety abnormal event occurs, acquire the determination result of the safety abnormal event, and broadcast the determination result of the safety abnormal event to all passing vehicles in the travel node coverage range to which the edge server belongs.

[0175] Embodiment five

[0176] Figure 4 The hardware structure diagram of the edge server provided in an embodiment of the present application is shown in FIG. 5. Figure 4 As shown in the figure, the edge server 50 comprises at least one processor 501 and a memory 502.

[0177] The memory 502 is configured to store computer execution instructions.

[0178] The processor 501 is configured to execute the computer execution instructions stored in the memory to realize each step executed in the above-mentioned embodiments. For details, please refer to the related description in the foregoing method embodiments.

[0179] Optionally, the memory 502 can be independent or integrated with the processor 501.

[0180] When the memory 502 is independently arranged, the edge server further comprises a bus 503 configured to connect the memory 502 and the processor 501.

[0181] Embodiment six

[0182] The embodiments of the present application further provide a computer readable storage medium, wherein computer execution instructions are stored in the computer readable storage medium, and when the processor executes the computer execution instructions, the vehicle control method as described above is realized.

[0183] Embodiment seven

[0184] The embodiments of the present application further provide a computer program product comprising a computer program, and when the processor executes the computer program, the vehicle control method as described above is realized.

[0185] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other manners. For example, the embodiments of the device described above are merely schematic. For example, the division of the modules is merely logical function division. There can be another division manner for the actual implementation. For example, a plurality of modules or a function can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or modules, and can be electrical, mechanical or in other forms.

[0186] The modules illustrated as separated components can or can not be physically separated, and the components illustrated as modules can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to implement the embodiments of the present application.

[0187] In addition, each function module in each embodiment of the present application can be integrated in one processing unit, or each module can be physically present separately, or two or more modules can be integrated in one unit. The unit of the above modules can be realized in the form of hardware, or in the form of hardware plus software function unit.

[0188] The integrated module realized in the form of software function module can be stored in a computer readable storage medium. The software function module stored in the storage medium includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute part of the steps of the method described in each embodiment of the present application.

[0189] It should be understood that the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor. The steps of the method disclosed in the present application can be directly embodied as a hardware processor to execute, or be executed by a combination of hardware and software modules in the processor.

[0190] The memory can include a high-speed RAM memory, and can also include a non-volatile storage NVM, such as at least one disk memory, and can also be a U disk, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0191] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0192] The storage medium can be realized by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0193] An exemplary storage medium is coupled to the processor so that the processor can read information from, and write information to, the storage medium. Of course, the storage medium can be part of the processor. The processor and the storage medium can be located in an application specific integrated circuits (ASIC). Of course, the processor and the storage medium can exist as discrete components in an electronic device or host device.

[0194] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims. Various modifications and changes can be made to the present application without departing from its scope.

[0195] It should be understood that the application is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made to the application without departing from the scope thereof. The scope of the application is limited only by the claims that follow.

Claims

1. A vehicle control method, characterized in that, An edge server deployed at any driving node along the driving route, including: Receive dynamic video of the vehicle's surroundings sent by at least one vehicle within the driving node, wherein the dynamic video of the vehicle's surroundings is captured by a camera device on the at least one vehicle. Receive static video of the road sent by the roadside equipment corresponding to the driving node; Based on the dynamic video of the vehicle's surroundings and the static video of the road, it is determined whether a safety anomaly has occurred; If a security anomaly is determined to have occurred, the determination result of the security anomaly is obtained and broadcast to all passing vehicles within the coverage area of ​​the driving node to which the edge server belongs. Also includes: The relationship between video content understanding accuracy and latency on the edge server and vehicle end is optimized using the following formula to achieve the optimal configuration of video content understanding accuracy and latency: In the formula, A model representing the relationship between video content understanding accuracy and latency; Indicates bandwidth; Indicates the total time. Indicates each moment; Indicates the importance weight of roadside anomalies; This indicates the number of targets in the abnormal scene that were correctly detected. The mean of the average precision is represented by , where This indicates the probability that the vehicle correctly detects the target. This indicates the probability that the roadside equipment correctly detects the target; The ( ) function is used to represent the relationship model between video quality QP value and video bitrate, where Represents the relationship model on the vehicle side. Represents the relationship model of the roadside; Indicates the compensation factor; in, - 6 represents the constraint condition for the optimal configuration; This means that the bandwidth resources of both the vehicle-side and roadside equipment cannot exceed the total bandwidth resources allocated to the vehicle-to-infrastructure (V2I) system. This indicates the uplink bandwidth of each roadside device; This indicates the uplink transmission channel bandwidth at the vehicle end; This represents the total bandwidth resources allocated to the vehicle-to-infrastructure (V2I) system. 2 indicates that the bandwidth resources of both the vehicle-side and roadside equipment are at least greater than the transmission bandwidth threshold of the vehicle-to-infrastructure system. This indicates the transmission bandwidth threshold of the vehicle-to-infrastructure (V2I) system. 3 indicates that the transmission rates of both the vehicle-side and roadside equipment cannot exceed the theoretical maximum data rate allowed by Shannon's formula; among which, This indicates the transmission rate at the vehicle end; Indicates the transmission rate of the roadside equipment; This represents the theoretical maximum data rate allowed by Shannon's formula; 4 indicates that the computational delay during transmission and calculation cannot exceed the total delay for completing the target detection task; whereby... This represents the computational delay during the transmission and computation process; This indicates the total latency requirement for completing the target detection task; 5 indicates that the total energy consumption of the video content understanding task cannot exceed the energy consumption limit during the task unloading process; among which, This represents the total energy consumption of the video content understanding task; Indicates the energy consumption limit during the task unloading process; 6 indicates that the vehicle-side target detection accuracy threshold and the edge server target detection accuracy threshold are both greater than the minimum threshold for video target detection accuracy; among which... This indicates the threshold for target detection accuracy at the vehicle end; This represents the target detection accuracy threshold for the edge server; This represents the minimum threshold for video target detection accuracy.

2. The method according to claim 1, characterized in that, The step of determining whether a safety anomaly has occurred based on the dynamic video of the vehicle's surroundings and the static video of the road includes: The relatively static scenes in the road still video are identified using a target detection algorithm; Based on the dynamic video surrounding the vehicle, basic road information is obtained; Based on the relatively static scene and the basic road information, determine whether a safety anomaly has occurred.

3. The method according to claim 2, characterized in that, The step of determining whether a safety anomaly has occurred based on the relatively static scene and the basic road information includes: The relatively static scene and the basic road information are input into the trained machine learning model to output the judgment result of the safety anomaly event.

4. The method according to claim 1, characterized in that, After receiving the dynamic video of the vehicle's surroundings sent by at least one vehicle within the driving node, the method further includes: The system receives the recognition result of a relatively static scene sent by the roadside device corresponding to the driving node; wherein the recognition result of the relatively static scene is obtained by the roadside device by matching the collected road still video with a preset data model. Based on the recognition results of the relatively static scene and the dynamic video around the vehicle, determine whether a safety anomaly has occurred; If a security anomaly is determined to have occurred, the determination result of the security anomaly is obtained and broadcast to all passing vehicles within the coverage area of ​​the driving node to which the edge server belongs.

5. A vehicle control device, characterized in that, An edge server deployed at any driving node along the driving route, including: The first receiving module is used to receive dynamic video of the vehicle's surroundings sent by at least one vehicle in the driving node, wherein the dynamic video of the vehicle's surroundings is collected by a camera device on the at least one vehicle. The second receiving module is used to receive static road video sent by the roadside equipment corresponding to the driving node; The first judgment module is used to determine whether a safety anomaly has occurred based on the dynamic video of the vehicle's surroundings and the static video of the road. The first broadcast module is used to obtain the determination result of the security anomaly if it is determined that a security anomaly has occurred, and broadcast the determination result of the security anomaly to all passing vehicles within the coverage area of ​​the driving node to which the edge server belongs. The relationship between video content understanding accuracy and latency on the edge server and vehicle end is optimized using the following formula to achieve the optimal configuration of video content understanding accuracy and latency: In the formula, A model representing the relationship between video content understanding accuracy and latency; Indicates bandwidth; Indicates the total time. Indicates each moment; Indicates the importance weight of roadside anomalies; This indicates the number of targets in the abnormal scene that were correctly detected. , represents the mean of the average precision, where This indicates the probability that the vehicle correctly detects the target. This indicates the probability that the roadside equipment correctly detects the target; The ( ) function is used to represent the relationship model between video quality (QP value) and video bitrate, where Represents the relationship model on the vehicle side. Represents the relationship model of the roadside; Indicates the compensation factor; in, - 6 represents the constraint condition for the optimal configuration; This means that the bandwidth resources of both the vehicle-side and roadside equipment cannot exceed the total bandwidth resources allocated to the vehicle-to-infrastructure (V2I) system. This indicates the uplink bandwidth of each roadside device; This indicates the uplink transmission channel bandwidth at the vehicle end; This represents the total bandwidth resources allocated to the vehicle-to-infrastructure (V2I) system. 2 indicates that the bandwidth resources of both the vehicle-side and roadside equipment are at least greater than the transmission bandwidth threshold of the vehicle-to-infrastructure system. This indicates the transmission bandwidth threshold of the vehicle-to-infrastructure (V2I) system. 3 indicates that the transmission rates of both the vehicle-side and roadside equipment cannot exceed the theoretical maximum data rate allowed by Shannon's formula; among which, This indicates the transmission rate at the vehicle end; Indicates the transmission rate of the roadside equipment; This represents the theoretical maximum data rate allowed by Shannon's formula; 4 indicates that the computational delay during transmission and calculation cannot exceed the total delay for completing the target detection task; whereby... This represents the computational delay during the transmission and computation process; This indicates the total latency requirement for completing the target detection task; 5 indicates that the total energy consumption of the video content understanding task cannot exceed the energy consumption limit during the task unloading process; among which, This represents the total energy consumption of the video content understanding task; Indicates the energy consumption limit during the task unloading process; 6 indicates that the vehicle-side target detection accuracy threshold and the edge server target detection accuracy threshold are both greater than the minimum threshold for video target detection accuracy; among which... This indicates the threshold for target detection accuracy at the vehicle end; This represents the target detection accuracy threshold for the edge server; This represents the minimum threshold for video target detection accuracy.

6. The apparatus according to claim 5, characterized in that: The first judgment module is specifically used to identify relatively static scenes in the road still video using a target detection algorithm; to obtain basic road information based on the dynamic video around the vehicle; and to determine whether a safety anomaly has occurred based on the relatively static scene and the basic road information.

7. The apparatus according to claim 6, characterized in that: The first judgment module is further specifically used to input the relatively static scene and the basic road information into the trained machine learning model to output the judgment result of the safety anomaly event.

8. An edge server, characterized in that, include: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the vehicle control method as described in any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the vehicle control method as described in any one of claims 1 to 4.

Images