Method, device and equipment for controlling thunder-vision vehicle detector and storage medium

Abstract

The invention provides a control method, device and equipment for a thunder-view vehicle detector and a storage medium, and relates to the technical field of intelligent transportation, and the method comprises the steps: obtaining image data collected by an image sensor and radar data collected by a radar in the thunder-view vehicle detector; determining a working condition characteristic factor according to the image data and the radar data; the working condition characteristic factors are used for representing the severity degree of the working condition of the thundersight vehicle detector; and adjusting the sensitivity of the radar according to the working condition characteristic factors. The method is suitable for being used in the detection process of a thundersight vehicle detector, and is used for solving the problem of low precision under severe working conditions.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a control method, device, equipment and storage medium of a radar vehicle detector. Background Art

[0002] Video surveillance plays a major role in traffic management. However, in real environments, harsh scenarios such as lighting changes, complex target motion, occlusion, foreground and background color interference, and background clutter may cause false alarms or missed alarms when using visual algorithms to analyze surveillance videos.

[0003] Millimeter-wave radar performs well in terms of speed and distance measurement accuracy. Most existing technologies can use radar data collected by millimeter-wave radar to compensate for false positives or missed positives caused by the use of visual algorithms alone to detect surveillance videos, thereby improving the accuracy and robustness of target detection.

[0004] However, in some cases, there may be a serious lack of visual signals in the surveillance video, which affects the accuracy of target detection by the radar-assisted vehicle inspection device that relies mainly on vision. Summary of the invention

[0005] Based on the above technical problems, the present application provides a radar vehicle detector control method, device, equipment and storage medium, which can adjust the sensitivity of the radar according to the working conditions of the radar vehicle detector, so that the radar vehicle detector adapts to the current working conditions and improves the accuracy of target detection.

[0006] In a first aspect, the present application provides a method for controlling a radar-visual vehicle detector, the method comprising: acquiring image data collected by an image sensor in the radar-visual vehicle detector and radar data collected by a radar; determining an operating condition characteristic factor based on the image data and the radar data; the operating condition characteristic factor is used to indicate the severity of the operating condition in which the radar-visual vehicle detector is located; and adjusting the sensitivity of the radar based on the operating condition characteristic factor.

[0007] In a possible implementation, determining an operating condition characteristic factor based on image data and radar data includes: determining visual reference distances corresponding to each of the multiple identified targets based on the image data; the visual reference distance is the distance between a first position and the position of a reference target; the first position is the position where the corresponding identified target is first identified based on the image data, or the position where the corresponding identified target is last identified based on the image data; determining radar reference distances corresponding to each of the multiple identified targets based on the radar data; the radar reference distance is the distance between a second position and the position of the reference target; the second position is the position where the corresponding identified target is first identified based on the radar data, or the position where the corresponding identified target is last identified based on the radar data; determining the operating condition characteristic factor based on the visual reference distances corresponding to each of the multiple identified targets and the radar reference distances corresponding to each of the multiple identified targets.

[0008] Optionally, determining the operating condition characteristic factor according to the visual reference distances corresponding to the plurality of identified targets and the radar reference distances corresponding to the plurality of identified targets includes: determining the operating condition characteristic factor according to the following formula:

[0009]

[0010] Among them, V r represents the working condition characteristic factor; v_dist represents the visual reference distance corresponding to multiple identified targets; r_dist represents the radar reference distance corresponding to multiple identified targets; E represents the mean function; k represents the coverage ratio coefficient of the visual action distance; σ represents the standard deviation function; Median represents the median function.

[0011] Optionally, the detection area of ​​the radar vehicle detector includes a single vision area and a radar vision fusion area; the single vision area is an area that detects and identifies targets using image data alone; the radar vision fusion area is an area that detects and identifies targets using image data and radar data; the distance between the single vision area and the radar vehicle detector is smaller than the distance between the radar vision fusion area and the radar vehicle detector; the visual reference distance includes the visual reference distance during the day and the visual reference distance at night; the method also includes: determining the visual distance during the day based on the visual reference distance during the day and the operating condition characteristic factor; the visual distance during the day is the distance between the third position and the reference target position; the third position is the position farthest from the reference target position in the single vision area during the day; determining the visual distance at night based on the visual reference distance at night and the operating condition characteristic factor; the visual distance at night is the distance between the fourth position and the reference target position; the fourth position is the position farthest from the reference target position in the single vision area at night.

[0012] In a possible implementation, determining the visual distance during the day according to the visual reference distance during the day and the operating condition characteristic factor includes: determining the visual distance during the day according to the following formula:

[0013] D v _day=S day +A×V r ;

[0014] Among them, D v _day indicates the visual distance during the day; S day represents the visual reference distance during the day; A represents the first preset coefficient; V r Indicates the characteristic factor of the working condition;

[0015] Determine the visual distance at night according to the visual reference distance at night and the working condition characteristic factor, including: determine the visual distance at night according to the following formula:

[0016] D v _night=S night +B×V r ;

[0017] Among them, D v _night indicates the visual distance at night; S night represents the visual reference distance at night; B represents the second preset coefficient.

[0018] Optionally, the operating condition characteristic factor is negatively correlated with the severity of the operating condition of the radar vehicle inspection device; the sensitivity of the radar is adjusted according to the operating condition characteristic factor, including: when the operating condition characteristic factor is greater than a preset threshold, the sensitivity of the radar is adjusted to a first sensitivity; when the operating condition characteristic factor is less than a preset threshold, the sensitivity of the radar is adjusted to a second sensitivity; the second sensitivity is higher than the first sensitivity.

[0019] It should be understood that image sensors usually convert optical signals into electrical signals, and convert electrical signals into image data. Therefore, when the working conditions of the radar vehicle detector are relatively bad, the image sensor is easily affected in the process of converting optical signals into electrical signals, resulting in the loss of visual signals in the image data. Radar obtains radar data by transmitting and receiving wireless signals, and is not easily affected by bad working conditions. Therefore, compared with image sensors, radar has better environmental applicability. The control method provided in the embodiment of the present application can determine the working condition characteristic factor according to the image data and radar data, and adjust the sensitivity of the radar according to the working condition characteristic factor, so that the radar vehicle detector can adapt to the current working conditions, significantly improve the detection performance of the radar vehicle detector in common bad working conditions, solve the low performance problem of the radar vehicle detector in bad working conditions, and improve the generalization of the use scenarios of the radar vehicle detector, thereby improving the accuracy of the radar vehicle detector in target detection as a whole.

[0020] In a second aspect, the present application provides a radar vehicle detector control device, which includes various functional modules used in the method described in the first aspect above.

[0021] In a third aspect, the present application provides an electronic device, which includes a processor and a memory; the memory stores instructions executable by the processor; when the processor is configured to execute the instructions, the electronic device implements the method described in the first aspect above.

[0022] In a fourth aspect, the present application provides a computer program product. When the computer program product is executed in an electronic device, the electronic device executes the related method described in the first aspect to implement the method described in the first aspect.

[0023] In a fifth aspect, the present application provides a readable storage medium, which includes: software instructions; when the software instructions are executed in an electronic device, the electronic device implements the method described in the first aspect above.

[0024] The beneficial effects of the second to fifth aspects mentioned above can be referred to the first aspect and will not be elaborated on again. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A schematic diagram of the composition of a control system provided in an embodiment of the present application;

[0027] Figure 2 A schematic diagram of the composition of an electronic device provided in an embodiment of the present application;

[0028] Figure 3 A flow chart of a control method provided in an embodiment of the present application;

[0029] Figure 4 Another schematic diagram of a control method according to an embodiment of the present application;

[0030] Figure 5 A schematic diagram of reference distance provided in an embodiment of the present application;

[0031] Figure 6 A schematic diagram of another flow chart of the control method provided in the embodiment of the present application;

[0032] Figure 7 A flowchart of a severe working condition trigger provided in an embodiment of the present application;

[0033] Figure 8 A schematic diagram of the composition of a control device provided in an embodiment of the present application;

[0034] Fig. 9 Another schematic diagram of the composition of the control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] First, the terms involved in the embodiments of the present application are introduced.

[0036] 1. Radar vehicle inspection device:

[0037] A road scene monitoring device that combines radar (such as millimeter wave radar or other types of radar, etc.) and image sensor. The radar vehicle detector can be used to collect traffic parameters and sequence traffic, and it can also be used in traffic light control, road safety warning, and vehicle-road collaborative control.

[0038] 2. Millimeter-wave radar (MWR):

[0039] A radar operating in the millimeter wave band (30-300GHz frequency domain, 1-10mm wavelength).

[0040] 3. Harsh working conditions:

[0041] Application scenarios where the performance of the radar vehicle inspection device is poor due to factors such as strong exposure, poor light, and bad weather.

[0042] 4. Other terms:

[0043] In the following, the terms "first", "second", "third", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", or "third", etc. may explicitly or implicitly include one or more of the features.

[0044] As described in the background art, in some cases (such as the above-mentioned severe working conditions), there may be a serious lack of visual signals in the monitoring video, affecting the accuracy of target detection by the radar-assisted vehicle inspection device that relies mainly on vision.

[0045] Based on this, the embodiments of the present application provide a control method, device, equipment and storage medium. The sensitivity of the (millimeter wave) radar can be adjusted according to the working condition of the radar vehicle detector, so that the radar vehicle detector adapts to the current working condition and improves the accuracy of target detection.

[0046] The following is an introduction with reference to the accompanying drawings.

[0047] Figure 1 This is a schematic diagram of the composition of the control system provided in the embodiment of the present application. Figure 1 As shown, the system may include: a radar vehicle detector 100 and a control device 200. The radar vehicle detector 100 and the control device 200 may be connected via a wired network or a wireless network.

[0048] The radar vehicle detector 100 may include an image sensor and a radar.

[0049] The image sensor may be a charge coupled device (CCD) sensor, or other types of sensors, etc. The embodiment of the present application does not limit this.

[0050] The radar may be a millimeter wave radar or other types of radars, etc. This embodiment of the present application does not limit this.

[0051] Image sensors can be used to collect image data.

[0052] Radar can be used to collect radar data.

[0053] In some embodiments, the radar vehicle inspection device 100 may also be used to send image data and radar data to the control device 200 .

[0054] As described above, the radar vehicle detector 100 and the control device 200 may be connected via a wired network or a wireless network, which may include one or more media or devices capable of transmitting image data and radar data from the radar vehicle detector 100 to the control device 200 .

[0055] In some embodiments, the wired network or wireless network may include one or more communication media that enable the radar detector 100 to directly send the image data and the radar data to the control device 200. In this embodiment, the radar detector 100 may modulate the image data and the radar data according to a communication standard (e.g., a wireless communication protocol) and send the modulated image data and the radar data to the control device 200. The one or more communication media may include wireless and wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines.

[0056] Alternatively, the one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (eg, the Internet).

[0057] Optionally, the one or more communication media may also include a router, a switch, a base station, or other devices that facilitate communication from the radar vehicle detector 100 to the control device 200 .

[0058] The control device 200 may be an electronic device having a computing function, such as a computer or a processor.

[0059] Among them, the server can be a single server, or it can be a server cluster composed of multiple servers. In some implementations, the server cluster can also be a distributed cluster. Optionally, the server can also be implemented on a cloud platform, for example, the cloud platform can include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, and a multi-cloud, etc., or any combination thereof. The embodiments of the present application are not limited to this.

[0060] The control device 200 can be used to adjust the sensitivity of the radar according to the image data and the radar data. The specific adjustment process can be referred to in the following embodiments, which will not be repeated here.

[0061] It should be noted that Figure 1 In the description, the radar visual vehicle inspection device 100 and the control device 200 are taken as independent devices as examples. Optionally, the radar visual vehicle inspection device 100 and the control device 200 can also be integrated into one. That is, the radar visual vehicle inspection device 100 or its corresponding functions, and the control device 200 or its corresponding functions can be integrated into one device. For example, a radar visual vehicle inspection device having a function of adjusting radar sensitivity according to image data and radar data. In this case, the control device 200 can also be understood as a component in the radar visual vehicle inspection device 100 or the radar visual vehicle inspection device 100. The embodiments of the present application are not limited to this.

[0062] The execution subject of the control method provided in the embodiment of the present application is the above-mentioned control device 200. As mentioned above, the control device 200 can be an electronic device with computing and processing functions, such as a computer, a server, a radar vehicle inspection device 100, or a component in the radar vehicle inspection device 100. Optionally, the control device 200 can also be an application (application, APP) with control function installed in the aforementioned electronic device; or, the control device 200 can also be a processor (such as a central processing unit (CPU)) in the aforementioned electronic device; or, the control device 200 can also be a software system or platform deployed in the aforementioned electronic device; or, the control device 200 can also be a functional module in the aforementioned electronic device for executing the control method, etc. The embodiment of the present application does not limit this.

[0063] For simplicity of description, the control device 200 is taken as an electronic device as an example for description below.

[0064] Figure 2 The following is a schematic diagram of the composition of an electronic device provided in an embodiment of the present application. Figure 2As shown, the electronic device may include: a processor 10 , a memory 20 , a communication line 30 , a communication interface 40 , and an input / output interface 50 .

[0065] The processor 10 , the memory 20 , the communication interface 40 and the input / output interface 50 may be connected via a communication line 30 .

[0066] The processor 10 is used to execute the instructions stored in the memory 20 to implement the control method provided in the following embodiments of the present application. The processor 10 can be a CPU, a general-purpose processor network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller (MCU) / single-chip microcomputer, a programmable logic device (PLD), or any combination thereof. The processor 10 can also be any other device with processing functions, such as a circuit, a device, or a software module, which is not limited in the embodiments of the present application. In one example, the processor 10 may include one or more CPUs, such as Figure 2 As an optional implementation, the electronic device may include multiple processors, for example, in addition to the processor 10, it may also include a processor 60 ( Figure 2 The dashed line is used as an example.

[0067] The memory 20 is used to store instructions executable by the processor 10. For example, the instructions may be computer programs. Optionally, the memory 20 may be a read-only memory (ROM) or other types of static storage devices that can store static information and / or instructions, or a random access memory (RAM) or other types of dynamic storage devices that can store information and / or instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage devices, etc., and the embodiments of the present application are not limited to this.

[0068] It should be noted that the memory 20 may exist independently of the processor 10, or may be integrated with the processor 10. The memory 20 may be located inside the electronic device, or may be located outside the electronic device, which is not limited in the embodiment of the present application.

[0069] The communication line 30 is used to transmit information between various components included in the electronic device.

[0070] The communication interface 40 is used to communicate with other devices (such as the above-mentioned radar vehicle inspection device 100) or other communication networks. The other communication networks can be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 40 can be a module, a circuit, a transceiver or any device capable of achieving communication.

[0071] The input / output interface 50 is used to implement human-computer interaction between a user and an electronic device, for example, to implement action interaction or information interaction between a user and an electronic device.

[0072] Exemplarily, the input / output interface 50 may be a mouse, a keyboard, a display screen, or a touch display screen, etc. Action interaction or information interaction between a user and an electronic device may be achieved through a mouse, a keyboard, a display screen, or a touch display screen, etc.

[0073] It should be noted that Figure 2 The structure shown in the figure does not constitute a limitation on the electronic device, except Figure 2 In addition to the components shown, the electronic device may include more or fewer components than shown (eg, only the processor 10 and the memory 20 ), or a combination of certain components, or a different arrangement of components.

[0074] The control method provided in the embodiments of the present application is introduced below.

[0075] Figure 3 The following is a flow chart of the control method provided in the embodiment of the present application. Figure 3 As shown, the method includes S101 to S103.

[0076] S101. The electronic device obtains image data collected by an image sensor in a radar vehicle inspection device and radar data collected by a radar.

[0077] Optionally, as above Figure 1 As shown, the electronic device (control device 200 ) and the radar vehicle detector 100 are independent devices. In this case, the electronic device can obtain image data and radar data by receiving image data and radar data sent by the radar vehicle detector 100 .

[0078] Optionally, as described above, the electronic device (control device 200) may also be integrated into one body. In this case, the electronic device may directly obtain the image data collected by the image sensor and the radar data collected by the radar.

[0079] S102: The electronic device determines a working condition characteristic factor according to the image data and the radar data.

[0080] Among them, the working condition characteristic factor is used to indicate the severity of the working condition of the radar vehicle inspection device.

[0081] The specific process of S102 can be referred to as follows Figure 4 S1021 to S1023 in the description will not be repeated here.

[0082] S103. The electronic equipment adjusts the sensitivity of the radar according to the characteristic factors of the working condition.

[0083] The specific process of S103 can be referred to the description in the following embodiment, which will not be repeated here.

[0084] It should be understood that image sensors usually convert optical signals into electrical signals, and convert electrical signals into image data. Therefore, when the working conditions of the radar vehicle detector are relatively bad, the image sensor is easily affected in the process of converting optical signals into electrical signals, resulting in the loss of visual signals in the image data. Radar obtains radar data by transmitting and receiving wireless signals, and is not easily affected by bad working conditions. Therefore, compared with image sensors, radar has better environmental applicability. In the control method provided in the embodiment of the present application, the electronic device can determine the working condition characteristic factor according to the image data and the radar data, and adjust the sensitivity of the radar according to the working condition characteristic factor, so that the radar vehicle detector can adapt to the current working conditions, significantly improve the detection performance of the radar vehicle detector in common bad working conditions, solve the low performance problem of the radar vehicle detector in bad working conditions, and improve the generalization of the use scenarios of the radar vehicle detector, thereby improving the accuracy of the radar vehicle detector in target detection as a whole.

[0085] The specific process of the above S102 is introduced below.

[0086] In some possible embodiments, the electronic device may determine the operating condition characteristic factor based on the distance between the identification target and the reference target identified for the first time. In this case, Figure 4 Another flow chart of the control method provided in the embodiment of the present application. Figure 4 As shown, the above S102 may specifically include S1021 to S1023.

[0087] S1021. The electronic device determines the visual reference distances corresponding to the multiple recognition targets respectively according to the image data.

[0088] The recognition target may be any one or more of a car, a motorcycle, and an electric vehicle. The visual reference distance is the distance between the first position and the position of the reference target, and the first position is the position where the corresponding recognition target is first recognized according to the image data, or the position where the corresponding recognition target is last recognized according to the image data. The reference target may be a lane stop line or other fixed reference objects.

[0089] For example, taking the radar vehicle inspection device as a reference, for the incoming identification target, the electronic device can use the position where the corresponding identification target is first identified based on the image data as the first position; for the outgoing identification target, the electronic device can use the position where the corresponding identification target is last identified as the first position.

[0090] Optionally, the electronic device may determine the visual tracks corresponding to the multiple recognition targets respectively according to the image data, and use the position of the initial point or the position of the end point in the visual track as the first position.

[0091] Optionally, after determining the visual trajectory, the electronic device may also determine whether the visual trajectory is a valid trajectory.

[0092] For example, the electronic device may determine whether the visual track is continuous. If it is continuous, it is determined to be a valid track; if it is not continuous, it is determined to be an invalid track.

[0093] Optionally, after determining the visual reference distance, the electronic device may also determine whether the visual reference distance is a valid distance. If so, the electronic device may retain the visual reference distance; if not, the electronic device may delete the visual reference distance.

[0094] For example, the electronic device can specifically determine whether the visual reference distance is within the valid range. If the visual reference distance is within the valid range, the electronic device can determine that the visual reference distance is a valid distance; if the visual reference distance is outside the valid range, the electronic device can determine that the visual reference distance is not a valid distance.

[0095] The valid interval may be preset in the electronic device by the administrator. For example, the valid interval may be (0m, 1000m) or (0m, 1200m), etc. The embodiment of the present application does not limit the specific range of the valid interval.

[0096] S1022. The electronic device determines the radar reference distances corresponding to the multiple identified targets respectively according to the radar data.

[0097] Among them, the radar reference distance is the distance between the second position and the position of the reference target, and the second position is the position where the corresponding identification target is first identified according to the radar data or the position where the corresponding identification target is last identified according to the radar data.

[0098] For example, taking the radar vehicle inspection device as a reference, for the incoming target, the electronic device can use the position where the corresponding target is first identified based on the radar data as the second position; for the outgoing target, the electronic device can use the position where the corresponding target is last identified as the second position.

[0099] Optionally, the electronic device may determine radar tracks corresponding to each of the multiple identified targets specifically based on the radar data, and use the position of the initial point or the position of the end point in the radar track as the first position.

[0100] Optionally, after determining the radar track, the electronic device may also determine whether the radar track is a valid track. For details, please refer to the above description of the visual track, which will not be repeated here.

[0101] Optionally, after determining the radar reference distance, the electronic device may also determine whether the radar reference distance is a valid distance. For details, please refer to the above description of the visual reference distance, which will not be repeated here.

[0102] For example, Figure 5 This is a schematic diagram of the reference distance provided in the embodiment of the present application. Figure 5 As shown, the electronic device may first identify the position of the identified target based on the image data or radar data ( Figure 5 The black dots are shown as an example) and the lane stop line ( Figure 5 The distance between the two (shown by the thick black line as an example) is used as the visual reference distance or the radar reference distance.

[0103] S1023. The electronic device determines a working condition characteristic factor according to the visual reference distances corresponding to the multiple identified targets and the radar reference distances corresponding to the multiple identified targets.

[0104] Optionally, assuming that the visual reference distance and the radar reference distance satisfy a Gaussian distribution, the electronic device may determine the operating condition characteristic factor specifically according to the following formula (1):

[0105]

[0106] In formula (1), V rIndicates the working condition characteristic factor. v_dist indicates the visual reference distance corresponding to each of the multiple identified targets. r_dist indicates the radar reference distance corresponding to each of the multiple identified targets. E indicates the mean function. k indicates the coverage ratio coefficient of the visual range, k can be 1, 2, or 3, etc. When k=3, it is the 3σ interval under the Gaussian distribution. σ indicates the standard deviation function. Median indicates the median function.

[0107] Optionally, after the electronic device obtains the visual reference distance and radar reference distance corresponding to each of the multiple identified targets in the above steps S1021 and S1022, it can also store the visual reference distance and radar reference distance corresponding to each of the multiple identified targets in a fixed-size buffer, and when the stored visual reference distance and radar reference distance exceed the fixed size, overwrite the historical visual reference distance and historical radar reference distance with the newly acquired visual reference distance and radar reference distance.

[0108] Optionally, when the space occupied by the visual reference distance and the radar reference distance reaches a preset ratio of the above-mentioned fixed size, the electronic device can trigger the execution of the above-mentioned S1023 to calculate the operating condition characteristic factor.

[0109] The preset ratio may be preset in the electronic device by a manager. For example, the preset ratio may be 1 / 2 or 1 / 3, etc. The embodiment of the present application does not limit the specific value of the preset ratio.

[0110] The specific process of the above S103 is introduced below.

[0111] In some possible embodiments, the working condition characteristic factor may be negatively correlated with the severity of the working condition of the radar visual vehicle detector. That is, the worse the working condition of the radar visual vehicle detector, the smaller the working condition characteristic factor, and the better the working condition of the radar visual vehicle detector, the larger the working condition characteristic factor. Based on this premise, when the working condition characteristic factor is greater than the preset threshold, the electronic device can adjust the sensitivity of the radar to the first sensitivity; when the working condition characteristic factor is less than the preset threshold, the electronic device can adjust the sensitivity of the radar to the second sensitivity.

[0112] The preset threshold value may be preset in the electronic device by the administrator. For example, the preset threshold value may be 0.85, 0.9, or 0.95, etc. The specific value of the preset threshold value in the embodiment of the present application is not limited. The second sensitivity is higher than the first sensitivity.

[0113] It should be noted that, when the operating condition characteristic factor is equal to the preset threshold, the electronic device can adjust the sensitivity of the radar to the first sensitivity or the second sensitivity. This embodiment of the present application does not limit this.

[0114] For example, the electronic device can use the difference between the operating condition characteristic factor and the preset threshold as the dependent variable, and input the dependent variable into the sign function. If the dependent variable is a positive number, the sign function can output a first sign (e.g., 1); if the dependent variable is a negative number, the sign function can output a second sign (e.g., 0). In this case, if the sign output by the sign function is the first sign, the electronic device can determine that the operating condition characteristic factor is greater than the preset threshold, and adjust the sensitivity of the radar to the first sensitivity; if the sign output by the sign function is the second sign, the electronic device can determine that the operating condition characteristic factor is less than the preset threshold, and adjust the sensitivity of the radar to the second sensitivity.

[0115] Optionally, the electronic device may output the symbol specifically according to the following formula (2):

[0116] f(V r )=sign(V r -T) Formula (2)

[0117] In formula (2), f(V r ) represents the sign of the output. T represents the preset threshold value. In the case of k=2 above, T can be set to 0.9. sign represents a sign function, which outputs a first sign when its dependent variable is a positive number and outputs a second sign when its dependent variable is a negative number.

[0118] For another example, the electronic device may also directly compare the operating condition characteristic factor with the preset threshold. If the operating condition characteristic factor is greater than the preset threshold, the electronic device adjusts the sensitivity of the radar to the first sensitivity; if the operating condition characteristic factor is less than the preset threshold, the electronic device adjusts the sensitivity of the radar to the second sensitivity.

[0119] In the control method provided in the embodiment of the present application, the electronic device can also improve the sensitivity of the radar and enhance the use of radar data under relatively harsh working conditions, thereby achieving better target detection performance under harsh working conditions.

[0120] In some other possible embodiments, the electronic device may also preset a correspondence between the operating condition characteristic factor interval and the radar sensitivity. In this case, the electronic device may determine the target sensitivity corresponding to the current operating condition characteristic factor according to the correspondence between the operating condition characteristic factor interval and the radar sensitivity and the operating condition characteristic factor obtained in S102 (for distinction, it may be referred to as the current operating condition characteristic factor), and adjust the radar to the target sensitivity.

[0121] Optionally, the electronic device can use the current operating condition characteristic factor as an index, traverse the correspondence between the operating condition characteristic factor interval and the radar sensitivity, and take the radar sensitivity corresponding to the interval where the current operating condition characteristic factor is located in the correspondence between the operating condition characteristic factor interval and the radar sensitivity as the target sensitivity.

[0122] For example, the corresponding relationship between the operating condition characteristic factor and the radar sensitivity may be shown in the following Table 1:

[0123] Table 1

[0124] Working condition characteristic factor interval Radar sensitivity Interval 1 Sensitivity 1 Interval 2 Sensitivity 2 Interval 3 Sensitivity 3

[0125] As shown in Table 1, the table may include operating condition characteristic factor interval items and radar sensitivity items. The operating condition characteristic factor interval items may include: interval 1, interval 2, and interval 3. The radar sensitivity items may include: sensitivity 1, sensitivity 2, and sensitivity 3. There is a corresponding relationship between interval 1 and sensitivity 1; there is a corresponding relationship between interval 2 and sensitivity 2; there is a corresponding relationship between interval 3 and sensitivity 3.

[0126] In some possible embodiments, the detection area of ​​the radar vehicle detector includes a single vision area and a radar vision fusion area. The single vision area is an area that detects and identifies targets using image data alone, and the radar vision fusion area is an area that detects and identifies targets using image data and radar data. The distance between the single vision area and the radar vehicle detector is smaller than the distance between the radar vision fusion area and the radar vehicle detector. The above-mentioned visual reference distance includes the visual reference distance during the day and the visual reference distance at night. The electronic device can adjust the range of the single vision area according to the visual reference distance. In this case, Figure 6 Another flow chart of the control method provided in the embodiment of the present application. Figure 6 As shown, the method may further include S201 to S202.

[0127] S201. The electronic device determines the daytime visual distance according to the daytime visual reference distance and the working condition characteristic factor.

[0128] The visual distance during the day is the distance between the third position and the reference target position, and the third position is the position farthest from the reference target position in the single visual area during the day.

[0129] Optionally, the electronic device may determine the visual distance during the day according to the following formula (3):

[0130] D v _day=S day +A×V r Formula (3)

[0131] In formula (3), D v _day indicates the visual distance during the day. dayrepresents the visual reference distance during the day. A represents the first preset coefficient, for example, A can be 35, 36, or 37. The embodiment of the present application does not limit the specific value of the first preset coefficient.

[0132] S202. The electronic device determines the visual distance at night according to the visual reference distance at night and the operating condition characteristic factor.

[0133] The visual distance at night is the distance between the fourth position and the position of the reference target, and the fourth position is the position farthest from the reference target position in the single visual area at night.

[0134] Optionally, the electronic device may determine the visual distance at night according to the following formula (4):

[0135] D v _night=S night +B×V r Formula (4)

[0136] In formula (4), D v _night indicates the visual distance at night. night represents the visual reference distance at night. B represents the second preset coefficient, for example, B can be 49, 50, or 51. The embodiment of the present application does not limit the specific value of the second preset coefficient.

[0137] In some possible embodiments, in some extremely harsh working conditions, the electronic device can adjust the radar-visual vehicle detector to a completely radar-dominated fusion strategy. That is, when the working condition characteristic factor is less than the first threshold, the electronic device can set the detection area of ​​the radar-visual vehicle detector to a single radar area, which is an area that uses radar data alone to detect and identify targets.

[0138] The first threshold value may also be preset in the electronic device by a manager, and the first threshold value is smaller than the above-mentioned preset threshold value.

[0139] Based on the understanding of the above embodiments, Figure 7 The flowchart of the severe working condition triggering provided in the embodiment of the present application is as follows. Figure 7As shown, the electronic device can first obtain the visual trajectory and the radar trajectory, and determine whether it is a valid trajectory, calculate the distance between the position of the initial point / vanishing point of the trajectory and the stop line (visual reference distance and radar reference distance), and then determine whether the distance between the position of the initial point / vanishing point of the trajectory and the stop line is valid, and use the buffer (visual buffer and radar buffer) to loop cache the visual reference distance and the radar reference distance, and then determine whether the storage space is half used. If so, calculate the operating condition characteristic factor according to the visual reference distance and the radar reference distance, and determine whether the operating condition characteristic factor is greater than a preset threshold. If so, adjust the radar sensitivity to 0, if not, adjust the radar sensitivity to 15, and finally adjust the fused trajectory. The specific process can be referred to the above method embodiment, and will not be repeated here.

[0140] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. It should be easy to realize that the technical goals in this field are combined with the units and algorithm steps of each example described in the embodiments disclosed in this article, and the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical goals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0141] In an exemplary embodiment, Figure 8 This is a schematic diagram of the composition of the control device provided in the embodiment of the present application. Figure 8 As shown, the control device may include: a severe working condition triggering algorithm module 801 and a dynamic lightning post-vision fusion strategy module 802.

[0142] The severe working condition triggering algorithm module 801 is used to obtain the visual trajectory and the radar trajectory, and calculate the visual reference distance and the radar reference distance according to the visual trajectory and the radar trajectory, calculate the difference factor of the visual & radar distribution as the working condition characteristic factor, and determine the result of the severe working condition classification according to the working condition characteristic factor and the preset threshold hyperparameter.

[0143] The dynamic radar post-visual fusion strategy module 802 is used to dynamically adjust the visual reference distance according to the working condition characteristic factor, adjust the radar sensitivity, and fuse the visual track and the radar track according to the radar post-visual fusion strategy when the result of the severe working condition classification is a severe working condition. The specific fusion process can be referred to in the related art and will not be repeated here.

[0144] In some embodiments, the above Figure 8 The functional modules in can also be merged or split into functional units. In this case, Fig. 9 Another schematic diagram of the composition of the control device provided in the embodiment of the present application. Fig. 9 As shown, the device may include: an acquisition unit 901 and a processing unit 902.

[0145] The acquisition unit 901 is used to respectively acquire image data collected by the image sensor in the radar vehicle inspection device and radar data collected by the radar.

[0146] The processing unit 902 is used to determine the working condition characteristic factor according to the image data and the radar data; the working condition characteristic factor is used to indicate the severity of the working condition of the radar vehicle inspection device; and adjust the sensitivity of the radar according to the working condition characteristic factor.

[0147] Optionally, the processing unit 902 is specifically used to determine the visual reference distances corresponding to each of the multiple identified targets according to the image data; the visual reference distance is the distance between the first position and the position of the reference target; the first position is the position where the corresponding identified target is first identified according to the image data, or the position where the corresponding identified target is last identified according to the image data; determine the radar reference distances corresponding to each of the multiple identified targets according to the radar data; the radar reference distance is the distance between the second position and the position of the reference target; the second position is the position where the corresponding identified target is first identified according to the radar data, or the position where the corresponding identified target is last identified according to the radar data; determine the operating condition characteristic factor according to the visual reference distances corresponding to each of the multiple identified targets and the radar reference distances corresponding to each of the multiple identified targets.

[0148] Optionally, the processing unit 902 is specifically configured to determine the operating condition characteristic factor according to the following formula:

[0149]

[0150] Among them, V r represents the working condition characteristic factor; v_dist represents the visual reference distance corresponding to multiple identified targets; r_dist represents the radar reference distance corresponding to multiple identified targets; E represents the mean function; k represents the coverage ratio coefficient of the visual action distance; σ represents the standard deviation function; Median represents the median function.

[0151] Optionally, the detection area of ​​the radar vehicle detector includes a single vision area and a radar vision fusion area; the single vision area is an area that detects and identifies targets using image data alone; the radar vision fusion area is an area that detects and identifies targets using image data and radar data; the distance between the single vision area and the radar vehicle detector is smaller than the distance between the radar vision fusion area and the radar vehicle detector; the visual reference distance includes the visual reference distance during the day and the visual reference distance at night; the processing unit 902 is also used to determine the visual distance during the day based on the visual reference distance during the day and the operating condition characteristic factor; the visual distance during the day is the distance between the third position and the reference target position; the third position is the position farthest from the reference target position in the single vision area during the day; the visual distance at night is determined based on the visual reference distance at night and the operating condition characteristic factor; the visual distance at night is the distance between the fourth position and the reference target position; the fourth position is the position farthest from the reference target position in the single vision area at night.

[0152] Optionally, the processing unit 902 is specifically configured to determine the visual distance during the day according to the following formula:

[0153] D v _day=S day +A×V r ;

[0154] Among them, D v _day indicates the visual distance during the day; S day represents the visual reference distance during the day; A represents the first preset coefficient; V r Indicates the characteristic factor of the working condition;

[0155] Determine the visual distance at night using the following formula:

[0156] D v _night=S night +B×V r ;

[0157] Among them, D v _night indicates the visual distance at night; S night represents the visual reference distance at night; B represents the second preset coefficient.

[0158] Optionally, the operating condition characteristic factor is negatively correlated with the severity of the operating condition of the radar vehicle inspection device; the processing unit 902 is specifically used to adjust the sensitivity of the radar to a first sensitivity when the operating condition characteristic factor is greater than a preset threshold; when the operating condition characteristic factor is less than a preset threshold, adjust the sensitivity of the radar to a second sensitivity; the second sensitivity is higher than the first sensitivity.

[0159] It should be noted that Fig. 9The division of units in the above is schematic and is only a logical function division. There may be other divisions in actual implementation. For example, two or more functions may be integrated into one processing unit. The above integrated units may be implemented in the form of hardware or software functional units.

[0160] In an exemplary embodiment, the embodiment of the present application further provides a readable storage medium, including software instructions, which, when executed on an electronic device, enables the electronic device to execute any one of the methods provided in the above embodiments.

[0161] In an exemplary embodiment, the embodiment of the present application further provides a computer program product including computer-executable instructions, which, when executed on an electronic device, enables the electronic device to execute any one of the methods provided in the above embodiments.

[0162] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer-executable instructions. When the computer-executable instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer-executable instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer-executable instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more servers that can be integrated with the medium. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), etc.

[0163] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other changes to the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0164] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0165] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for controlling a radar vehicle detector, characterized in that: The method comprises: Obtain image data collected by the image sensor in the radar vehicle inspection device and radar data collected by the radar; Determine a working condition characteristic factor according to the image data and the radar data; the working condition characteristic factor is used to indicate the severity of the working condition of the radar vision vehicle inspection device; The sensitivity of the radar is adjusted according to the operating condition characteristic factor.

2. The method according to claim 1, characterized in that: The determining of the operating condition characteristic factor according to the image data and the radar data comprises: Determine visual reference distances corresponding to the plurality of recognition targets respectively according to the image data; the visual reference distance is the distance between the first position and the position of the reference target; the first position is the position where the corresponding recognition target is first recognized according to the image data, or the position where the corresponding recognition target is last recognized according to the image data; Determine the radar reference distances corresponding to the multiple identified targets respectively according to the radar data; the radar reference distance is the distance between the second position and the position of the reference target; the second position is the position where the corresponding identified target is first identified according to the radar data, or the position where the corresponding identified target is last identified according to the radar data; The operating condition characteristic factor is determined according to the visual reference distances corresponding to each of the plurality of identified targets and the radar reference distances corresponding to each of the plurality of identified targets.

3. The method according to claim 2, characterized in that The determining the operating condition characteristic factor according to the visual reference distances corresponding to the plurality of identified targets and the radar reference distances corresponding to the plurality of identified targets comprises: Determine the characteristic factor of the operating condition according to the following formula: Among them, V r represents the working condition characteristic factor; v_dist represents the visual reference distance corresponding to each of the multiple identified targets; r_dist represents the radar reference distance corresponding to each of the multiple identified targets; E represents the mean function; k represents the coverage ratio coefficient of the visual action distance; σ represents the standard deviation function; Median represents the median function.

4. The method according to claim 2 or 3, characterized in that: The detection area of ​​the radar vehicle detector includes a single vision area and a radar vision fusion area; the single vision area is an area that detects and identifies targets using the image data alone; the radar vision fusion area is an area that detects and identifies targets using the image data and the radar data; the distance between the single vision area and the radar vehicle detector is less than the distance between the radar vision fusion area and the radar vehicle detector; the visual reference distance includes a daytime visual reference distance and a nighttime visual reference distance; The method further comprises: Determine the visual distance during the day according to the visual reference distance during the day and the operating condition characteristic factor; the visual distance during the day is the distance between the third position and the reference target position; the third position is the position in the single visual area that is farthest from the reference target position during the day; The visual distance at night is determined according to the visual reference distance at night and the operating condition characteristic factor; the visual distance at night is the distance between the fourth position and the position of the reference target; the fourth position is the position farthest from the reference target position in the single visual area at night.

5. The method according to claim 4, characterized in that The determining the daytime visual distance according to the daytime visual reference distance and the operating condition characteristic factor comprises: The daytime visual distance is determined according to the following formula: D v _day=S day +A×V r : Among them, D v _day represents the visual distance during the day; S day represents the visual reference distance during the day; A represents the first preset coefficient; V r represents the characteristic factor of the working condition; The determining of the visual distance at night according to the visual reference distance at night and the operating condition characteristic factor comprises: The visual distance at night is determined according to the following formula: D v _night=S night +B×V r ; Among them, D v _night represents the visual distance of the night; S night represents the visual reference distance in the dark night; B represents the second preset coefficient.

6. The method according to any one of claims 1 to 3, characterized in that: The working condition characteristic factor is negatively correlated with the severity of the working condition of the radar vehicle detector; and adjusting the sensitivity of the radar according to the working condition characteristic factor includes: When the operating condition characteristic factor is greater than a preset threshold, adjusting the sensitivity of the radar to a first sensitivity; When the operating condition characteristic factor is less than the preset threshold, the sensitivity of the radar is adjusted to a second sensitivity; the second sensitivity is higher than the first sensitivity.

7. A control device for a radar vehicle inspection device, characterized in that: The device comprises: an acquisition unit and a processing unit; The acquisition unit is used to respectively acquire image data collected by the image sensor in the radar vehicle inspection device and radar data collected by the radar; The processing unit is used to determine a working condition characteristic factor according to the image data and the radar data; the working condition characteristic factor is used to indicate the severity of the working condition of the radar vehicle detector; and the sensitivity of the radar is adjusted according to the working condition characteristic factor.

8. The device according to claim 7, characterized in that The processing unit is specifically used to determine the visual reference distances corresponding to each of the multiple identified targets according to the image data; the visual reference distance is the distance between a first position and the position of a reference target; the first position is the position where the corresponding identified target is first identified according to the image data, or the position where the corresponding identified target is last identified according to the image data; determine the radar reference distances corresponding to each of the multiple identified targets according to the radar data; the radar reference distance is the distance between a second position and the position of the reference target; the second position is the position where the corresponding identified target is first identified according to the radar data, or the position where the corresponding identified target is last identified according to the radar data; determine the operating condition characteristic factor according to the visual reference distances corresponding to each of the multiple identified targets and the radar reference distances corresponding to each of the multiple identified targets; and / or, The processing unit is specifically used to determine the operating condition characteristic factor according to the following formula: Among them, V r represents the working condition characteristic factor; v_dist represents the visual reference distance corresponding to each of the multiple identified targets; r_dist represents the radar reference distance corresponding to each of the multiple identified targets; E represents the mean function; k represents the coverage ratio coefficient of the visual range; σ represents the standard deviation function; Median represents the median function; and / or, The detection area of ​​the radar vision vehicle detector includes a single vision area and a radar vision fusion area; the single vision area is an area that detects and identifies targets using the image data alone; the radar vision fusion area is an area that detects and identifies targets using the image data and the radar data; the distance between the single vision area and the radar vision vehicle detector is less than the distance between the radar vision fusion area and the radar vision vehicle detector; the visual reference distance includes a daytime visual reference distance and a nighttime visual reference distance; the processing unit is further used to determine the daytime visual distance based on the daytime visual reference distance and the operating condition characteristic factor; the daytime visual distance is the distance between the third position and the reference target position; the third position is the position farthest from the reference target position in the single vision area during the day; the nighttime visual distance is determined based on the nighttime visual reference distance and the operating condition characteristic factor; the nighttime visual distance is the distance between the fourth position and the reference target position; the fourth position is the position farthest from the reference target position in the single vision area during the night; and / or, The processing unit is specifically used to determine the daytime visual distance according to the following formula: D v _day=S day +A×V r : Among them, D v _day represents the visual distance during the day; S day represents the visual reference distance during the day; A represents the first preset coefficient; V r represents the characteristic factor of the working condition; The visual distance at night is determined according to the following formula: D v _night=S night +B×V r ; Among them, D v _night represents the visual distance of the night; S night represents the visual reference distance at night; B represents the second preset coefficient; and / or, The operating condition characteristic factor is negatively correlated with the severity of the operating condition of the radar vehicle inspection device; the processing unit is specifically used to adjust the sensitivity of the radar to a first sensitivity when the operating condition characteristic factor is greater than a preset threshold; and to adjust the sensitivity of the radar to a second sensitivity when the operating condition characteristic factor is less than the preset threshold; the second sensitivity is higher than the first sensitivity.

9. An electronic device, characterized in that: include: Processor and memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the electronic device implements the method according to any one of claims 1 to 6.

10. A readable storage medium, characterized in that: The readable storage medium includes: software instructions; When the software instructions are executed in an electronic device, the electronic device implements the method according to any one of claims 1 to 6.

Images