Vehicle alert method and apparatus, program product, and storage medium

Abstract

A vehicle alert method and apparatus, a program product, and a storage medium. The method comprises: acquiring an image of the surrounding environment of an ego vehicle; and if a perception network determines that high beams are used by an oncoming vehicle, outputting alert information.

Description

A vehicle alarm method, device, program product, and storage medium Technical Field

[0001] This application relates to the field of intelligent driving, specifically to a vehicle alarm method, device, program product, and storage medium. Background Technology

[0002] Since its invention, the automobile has undergone continuous development and has become an indispensable part of people's lives. With the development of artificial intelligence technology, automobiles are also evolving towards intelligence, giving rise to autonomous driving technology.

[0003] However, during autonomous driving, if oncoming vehicles are using high beams when meeting each other, the high beams can cause glare, which can lead to abnormal recognition by the vehicle's camera, causing the camera to malfunction and affecting the vehicle's perception system, thus impacting driving safety. Summary of the Invention

[0004] This application provides a vehicle alarm method, device, program product, and storage medium to provide timely alarms for glare caused by the high beams of oncoming vehicles, thereby ensuring driving safety.

[0005] In view of this, in a first aspect, embodiments of this application provide a vehicle warning method, the method comprising: acquiring an image of the surrounding environment of a vehicle; and if a perception network determines, based on the image of the surrounding environment of the vehicle, that an oncoming vehicle is using high beams, outputting warning information. The perception network is used to detect the lighting usage of oncoming vehicles based on the image of the surrounding environment of the vehicle.

[0006] In the method provided by this application, the surrounding environment image of the vehicle is identified and detected by a perception network. When it is determined that an oncoming vehicle is using high beams, a warning message will be output in a timely manner. In this way, a warning can be issued in a timely manner for the glare caused by the high beams of the oncoming vehicle, so that the driver can take over the vehicle in time, or the intelligent driving system can adjust the driving strategy in time, thereby reducing the impact of camera failure on vehicle driving and ensuring driving safety.

[0007] In conjunction with the first aspect, in one possible implementation of the first aspect, the perception network includes: a first subnetwork and a second subnetwork, the first subnetwork being used to detect the headlight position of an oncoming vehicle based on an image of the vehicle's surrounding environment, and the second subnetwork being used to detect the headlight usage of the oncoming vehicle based on the detection results of the first subnetwork and the image of the vehicle's surrounding environment; correspondingly, determining that the oncoming vehicle is using high beams includes: inputting the image of the vehicle's surrounding environment into the first subnetwork for target detection to obtain first headlight position information corresponding to the oncoming vehicle in the image of the vehicle's surrounding environment; inputting the image of the vehicle's surrounding environment and the first headlight position information into the second subnetwork to determine that the oncoming vehicle is using high beams.

[0008] In the method provided by the embodiments of this application, two specific sub-networks are designed so that the headlight position information of the oncoming vehicle can be determined first, and then the headlight position information of the oncoming vehicle can be combined to determine the lighting usage of the oncoming vehicle. In this way, combined with the headlight position of the high beam, the lighting usage of the oncoming vehicle can be determined more accurately, and a warning message can be output in a timely manner when the oncoming vehicle uses the high beam.

[0009] In conjunction with the first aspect, in one possible implementation of the first aspect, inputting an image of the vehicle's surrounding environment and the position information of the first headlight into a second sub-network to determine that an oncoming vehicle is using high beams includes: inputting an image of the vehicle's surrounding environment and the position information of the first headlight into the second sub-network; dividing the image of the vehicle's surrounding environment through the second sub-network to obtain multiple local images corresponding to the image of the vehicle's surrounding environment; and determining that an oncoming vehicle is using high beams based on the position information of the first headlight and the image quality scores corresponding to the local images.

[0010] In the method provided in this application, by combining the headlight position information of the oncoming vehicle and the image quality score corresponding to the local image, the image quality score corresponding to the headlight area of ​​the oncoming vehicle can be determined. Moreover, the image quality score corresponding to the headlight area when the headlight is emitting high beams is significantly different from the image quality score corresponding to the headlight area when the headlight is not emitting high beams. Thus, by using the headlight position information of the oncoming vehicle and multiple image quality scores, the lighting usage of the oncoming vehicle can be determined more accurately, and a warning message can be output in a timely manner when the oncoming vehicle uses high beams.

[0011] In conjunction with the first aspect, in one possible implementation of the first aspect, determining whether an oncoming vehicle is using high beams based on the first headlight position information and image quality score includes: determining a center position based on the first headlight position information; determining a quality average based on the image quality scores corresponding to local images around the center position; and determining that the quality average is less than a quality threshold to confirm that the oncoming vehicle is using high beams. This method can determine the quality average corresponding to the headlight area of ​​the oncoming vehicle, and by comparing it with the quality threshold, it can more accurately determine the headlight usage of the oncoming vehicle and promptly output warning information when the oncoming vehicle uses high beams.

[0012] In conjunction with the first aspect, in one possible implementation of the first aspect, the vehicle's surrounding environment image is presented in multiple frames. The output of alarm information includes: if the perception network determines, based on the multiple frames of the vehicle's surrounding environment image, that an oncoming vehicle is using high beams in N consecutive frames of the vehicle's surrounding environment image, then outputting alarm information, where N is a positive integer and N is greater than 1. This ensures that alarm information is only output when oncoming vehicles are confirmed to be using high beams in multiple consecutive frames of the vehicle's surrounding environment image, avoiding the possibility of brief misjudgments due to external environmental interference leading to erroneous alarm information output, thus enabling more accurate alarm information output.

[0013] In conjunction with the first aspect, in one possible implementation of the first aspect, the method further includes: inputting a historical image of the vehicle's surrounding environment into a first training network to obtain second headlight position information; inputting the historical image of the vehicle's surrounding environment and the second headlight position information into a second training network to obtain high beam status information, wherein the high beam status information is used to characterize whether oncoming vehicles in the historical image of the vehicle's surrounding environment are using high beams; determining a first loss value using the second headlight position information and the headlight position annotation information; determining a second loss value using the high beam status information and the high beam status annotation information; and iteratively training the first training network and the second training network using the first loss value and the second loss value to obtain a perception network including a first sub-network and a second sub-network.

[0014] In the method provided by this application, a perception network including a first sub-network and a second sub-network is trained using historical images of the vehicle's surrounding environment. In practical applications, the trained perception network can be used to determine the lighting usage of oncoming vehicles and to issue timely warnings for glare caused by the high beams of oncoming vehicles, thus ensuring driving safety.

[0015] In conjunction with the first aspect, in one possible implementation of the first aspect, the method further includes: issuing a voice reminder message, and / or displaying a text reminder message, and / or identifying oncoming vehicles using high beams on the vehicle's interactive interface, and / or vibrating the steering wheel; and controlling the vehicle's high beams and low beams to flash alternately. This not only reminds the driver of the vehicle that oncoming vehicles are using high beams and need to take over, but also politely prompts the drivers of oncoming vehicles to turn off their high beams, thereby further ensuring driving safety.

[0016] In conjunction with the first aspect, in one possible implementation of the first aspect, the alarm information is used to warn of camera failure in the vehicle, and the method further includes: controlling the vehicle to decelerate and move to the right. Thus, in the event of camera failure in the vehicle and the driver's inability to take over in time, the intelligent driving system can promptly adjust the control strategy to control the vehicle to decelerate and move to the right, ensuring driving safety.

[0017] Secondly, embodiments of this application provide a vehicle warning device, the device comprising:

[0018] The acquisition module acquires images of the vehicle's surrounding environment.

[0019] The alarm module is used to output alarm information if the perception network determines that an oncoming vehicle is using high beams based on the image of the vehicle's surrounding environment; the perception network is used to detect the lighting status of oncoming vehicles based on the image of the vehicle's surrounding environment.

[0020] The vehicle alarm device has the function of implementing the vehicle alarm method in the first aspect or any possible embodiment of the first aspect. This function can be implemented by hardware or by hardware executing corresponding software, and the hardware or software includes one or more modules corresponding to the above function.

[0021] The beneficial effects shown in this aspect are similar to those in the first aspect, as detailed in the first aspect, and will not be repeated here.

[0022] Thirdly, embodiments of this application provide a vehicle alarm device, which may include at least one processor, the processor being configured to invoke computer instructions in a memory to cause the vehicle alarm device to execute the vehicle alarm method in the first aspect or any optional embodiment of the first aspect.

[0023] In conjunction with the third aspect, in one possible implementation of the third aspect, the vehicle warning device may further include a memory.

[0024] Fourthly, embodiments of this application provide a computer-readable storage medium that may include instructions that, when executed on a computer, cause the computer to perform the vehicle alarm method of the first aspect or any optional embodiment of the first aspect.

[0025] Fifthly, embodiments of this application provide a computer program product that may include instructions that, when executed on a computer, cause the computer to perform the vehicle alarm method in the first aspect or any optional embodiment of the first aspect.

[0026] Sixthly, embodiments of this application provide a chip system including a processor for supporting a device in implementing the functions involved in the foregoing aspects, such as transmitting or processing data and / or information involved in the foregoing methods. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for the device. The chip system may be composed of chips or may include chips and other discrete devices.

[0027] In a seventh aspect, embodiments of this application provide a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to execute the vehicle alarm method in the first aspect or any optional embodiment of the first aspect.

[0028] Eighthly, embodiments of this application provide a vehicle that may include at least one processor, the processor being configured to invoke computer instructions in memory to cause a vehicle warning device to perform the vehicle method of the first aspect or any optional embodiment of the first aspect.

[0029] In conjunction with the eighth aspect, in one possible implementation of the eighth aspect, the vehicle may also include a memory. Attached Figure Description

[0030] Figure 1 is a schematic diagram of an application scenario of the vehicle alarm method provided in the embodiment of this application;

[0031] Figure 2 is a flowchart illustrating a vehicle alarm method provided in an embodiment of this application;

[0032] Figure 3 is a schematic diagram of an image of the environment surrounding a vehicle provided in an embodiment of this application;

[0033] Figure 4 is a flowchart illustrating another vehicle alarm method provided in an embodiment of this application;

[0034] Figure 5 is a schematic diagram of a first headlight position information provided in an embodiment of this application;

[0035] Figure 6 is a schematic diagram of the input and output of a sensing network provided in an embodiment of this application;

[0036] Figure 7 is a schematic diagram of a partial image provided in an embodiment of this application;

[0037] Figure 8 is a schematic diagram of a central location and a partial image around the central location provided in an embodiment of this application;

[0038] Figure 9 is a flowchart illustrating a training method for a perceptual network provided in an embodiment of this application.

[0039] Figure 10 is a schematic diagram of the composition structure of a vehicle alarm device provided in an embodiment of this application;

[0040] Figure 11 is a structural schematic diagram of another vehicle alarm device provided in an embodiment of this application;

[0041] Figure 12 is a structural schematic diagram of a vehicle provided in an embodiment of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions "a," "an," "the," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the embodiments of this application, "one or more" refers to one, two, or more; "and / or" describes the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0043] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0044] The "multiple" mentioned in the embodiments of this application refers to two or more. It should be noted that in the description of the embodiments of this application, terms such as "first" and "second" are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance, nor should they be construed as indicating or implying order.

[0045] During autonomous driving, if oncoming vehicles are using high beams when meeting each other, the glare can cause malfunctions in the vehicle's camera, affecting its perception system and leading to frequent false positives, missed detections, and delayed detections, thus compromising driving safety.

[0046] Currently, it's possible to determine whether an oncoming vehicle is using high beams by combining LiDAR sensors with pre-set vehicle information in a server. However, this method relies on LiDAR sensors and cannot be applied to vehicles without them; furthermore, it depends on pre-set vehicle information and cannot identify whether vehicles not pre-set in the server or modified vehicles are using high beams.

[0047] Therefore, this application provides a vehicle alarm method, device, program product, and storage medium, which uses a sensing network to detect the lighting usage of oncoming vehicles and promptly issues an alarm for glare caused by the high beams of oncoming vehicles, thereby ensuring driving safety.

[0048] Please refer to Figure 1, which is a schematic diagram of an application scenario for the vehicle warning method provided in this application. The vehicle warning method provided in this embodiment can be applied to scenarios involving oncoming traffic at night. A camera installed on the vehicle can capture images of the surrounding environment. A processor installed on the vehicle can acquire these images and input them into a perception network. The perception network detects the lighting usage of oncoming vehicles. When it is determined that an oncoming vehicle is using high beams, a warning message is promptly output to ensure driving safety. It is understood that the vehicle warning method provided in this embodiment can also be applied to other scenarios.

[0049] Specifically, please refer to Figure 2, which is a flowchart illustrating a vehicle alarm method provided in this application. This vehicle alarm method can be executed by a vehicle alarm device. The vehicle alarm method provided in this application embodiment may include the following steps:

[0050] S201. Obtain images of the surrounding environment of the vehicle.

[0051] The images of the vehicle's surrounding environment in this embodiment can be captured by a camera installed on the vehicle. The images of the vehicle's surrounding environment include environmental information such as oncoming vehicles, roads, green landscapes, etc.

[0052] Please refer to Figure 3, which is a schematic diagram of a vehicle's surrounding environment image provided in an embodiment of this application. The vehicle's surrounding environment image includes oncoming vehicles, etc., and the oncoming vehicles are using high beams.

[0053] S202. If the perception network determines that an oncoming vehicle is using high beams based on the image of the surrounding environment of the vehicle, it outputs a warning message.

[0054] The perception network in this embodiment is used to detect the lighting usage of oncoming vehicles based on images of the vehicle's surrounding environment. The perception network can be trained using historical images of the vehicle's surrounding environment. The lighting usage of oncoming vehicles can be any of the following: oncoming vehicles using high beams or oncoming vehicles not using high beams.

[0055] In this embodiment, after acquiring an image of the vehicle's surrounding environment, the image can be input into a perception network. The perception network then detects the lighting status of oncoming vehicles. If the perception network determines that an oncoming vehicle is using high beams, it can output an alarm message. This alarm message can be a voice alarm, a text alarm, or something similar.

[0056] In this embodiment, if the alarm information is a voice alarm, it can be played through the vehicle's speakers, such as "Camera malfunction," "Camera malfunction, please take over," "Oncoming vehicle using high beams," or "Perception system affected," etc. If the alarm information is a text alarm, it can be displayed on the vehicle's interactive interface, such as "Camera malfunction," "Camera malfunction, please take over," "Oncoming vehicle using high beams," or "Perception system affected," etc. In this embodiment, both voice alarms and text alarms can be played and displayed simultaneously.

[0057] In this embodiment, alarm information can also be output and reported to the intelligent driving system so that the intelligent driving system can adjust the driving strategy according to the alarm information, such as controlling the vehicle to slow down and drive to the right.

[0058] As can be seen, in this application, the sensor network is used to identify and detect the images of the vehicle's surrounding environment. When it is determined that an oncoming vehicle is using high beams, a warning message will be output in a timely manner. In this way, a warning can be issued in a timely manner for the glare caused by the high beams of oncoming vehicles, so that the driver can take over the vehicle in time, or the intelligent driving system can adjust the driving strategy in time, thereby reducing the impact of camera failure on vehicle driving and ensuring driving safety.

[0059] Please refer to Figure 4, which is a flowchart illustrating another vehicle alarm method provided in this application. Figure 4 provides a more detailed explanation of the vehicle alarm method based on the method provided in Figure 2. The vehicle alarm method provided in this application embodiment may include the following steps:

[0060] S401, Acquire images of the surrounding environment of the vehicle.

[0061] It is understood that S401 in this embodiment is the same as S201 in the above embodiment, so it will not be described again.

[0062] S402. Input the image of the surrounding environment of the vehicle into the first sub-network for target detection, and obtain the position information of the first headlight of the oncoming vehicle in the image of the surrounding environment of the vehicle.

[0063] In this embodiment, the perception network includes a first sub-network and a second sub-network. The first sub-network is used to detect the headlight positions of oncoming vehicles based on the surrounding environment image of the vehicle. The second sub-network is used to detect the headlight usage of oncoming vehicles based on the detection results of the first sub-network and the surrounding environment image of the vehicle. It is understood that the detection results of the first sub-network are the first headlight position information.

[0064] In the embodiments of this application, the first headlight position information may include multiple coordinate points corresponding to the paired headlights of the oncoming vehicle, 2D frames corresponding to the paired headlights of the oncoming vehicle, etc.

[0065] Please refer to Figure 5, which is a schematic diagram of a first headlight position information provided in an embodiment of this application. Figure 5 illustrates that the first headlight position information includes 2D frames corresponding to pairs of headlights of oncoming vehicles. In Figure 5, each headlight of an oncoming vehicle corresponds to a 2D frame, and the 2D frame includes the headlight of the oncoming vehicle, that is, the 2D frame can select the headlight area.

[0066] S403. Input the image of the vehicle's surrounding environment and the position information of the first headlight into the second sub-network.

[0067] In this embodiment, the second sub-network can take the output of the first sub-network and the image of the vehicle's surrounding environment as input, and the lighting usage of oncoming vehicles as output.

[0068] Please refer to Figure 6, which is a schematic diagram of the input and output of a perception network provided in an embodiment of this application. The perception network in Figure 6 uses an image of the vehicle's surrounding environment as input data, inputting the image into a first sub-network and a second sub-network. The first sub-network detects the headlight positions of oncoming vehicles in the surrounding environment image, outputs corresponding first headlight position information, and inputs this information into the second sub-network. The second sub-network, based on the first headlight position information and the surrounding environment image, detects the lighting usage of oncoming vehicles in the surrounding environment image, outputting whether the oncoming vehicle is using high beams or not. By designing two specific sub-networks, the headlight position information of oncoming vehicles can be determined first, and then combined with this information to determine the lighting usage of the oncoming vehicles. This allows for a more accurate determination of the oncoming vehicle's lighting usage by combining the headlight position of the high beams, enabling timely output of warning information when the oncoming vehicle is using high beams.

[0069] S404. The vehicle's surrounding environment image is divided by the second sub-network to obtain multiple local images corresponding to the vehicle's surrounding environment image.

[0070] The second sub-network in this embodiment has the following functions: image segmentation, image quality score determination, and determination of oncoming vehicle headlight usage, etc.

[0071] In this embodiment, the environment surrounding the vehicle can be divided into a w×h grid, and the image corresponding to each grid is a local image. Here, w is a positive integer, h is a positive integer, and w×h can be 40×20, 32×18, or 22×15, etc.

[0072] In one possible implementation, the present application can determine the value of w×h based on the size of the image of the vehicle's surrounding environment, that is, determine how many local images the image of the vehicle's surrounding environment needs to be divided into.

[0073] Please refer to Figure 7, which is a schematic diagram of a partial image provided in this application. In Figure 7, the image of the vehicle's surrounding environment is divided into 22×15 partial images.

[0074] S405. Based on the first headlight position information and the image quality score corresponding to the local image, determine that the oncoming vehicle is using high beams.

[0075] In this embodiment, a second sub-network can be used to score local images to obtain image quality scores for those local images. For example, image quality scores range from 0 to 1, with higher scores indicating higher image quality and lower scores indicating lower image quality. Local image 1 has an image quality score of 0.8, local image 2 has an image quality score of 0.5, and local image 3 has an image quality score of 0.9, indicating that local image 1 has higher image quality than local image 2, and local image 3 has lower image quality than local image 3. It is understood that the above is merely illustrative and should not be construed as limiting the embodiments of this application.

[0076] In this embodiment, by combining the headlight position information of the oncoming vehicle and the image quality score corresponding to the local image, the image quality score corresponding to the headlight area of ​​the oncoming vehicle can be determined. Moreover, the image quality score corresponding to the headlight area when the headlight is emitting high beams is significantly different from the image quality score corresponding to the headlight area when the headlight is not emitting high beams. Thus, by using the headlight position information of the oncoming vehicle and multiple image quality scores, the lighting usage of the oncoming vehicle can be determined more accurately, and a warning message can be output in a timely manner when the oncoming vehicle uses high beams.

[0077] In one possible implementation, embodiments of this application can determine the image quality score corresponding to each local image through a second sub-network to obtain multiple image quality scores. Then, based on the first headlight position information, the image quality score corresponding to the headlight of the oncoming vehicle is selected from the multiple image quality scores. Finally, based on the image quality score corresponding to the headlight of the oncoming vehicle, it is determined that the oncoming vehicle is using high beams.

[0078] In one possible implementation, the second sub-network in this application embodiment can select the local image corresponding to the headlight of the oncoming vehicle from multiple local images based on the first headlight position information, then determine the image quality score of the local image corresponding to the headlight of the oncoming vehicle, and then determine that the oncoming vehicle is using high beams based on the image quality score of the local image corresponding to the headlight of the oncoming vehicle.

[0079] In one possible implementation, S405 in this embodiment may include:

[0080] a1. Determine the center position based on the position information of the first headlight.

[0081] Please refer to Figure 8, which is a schematic diagram of a central position and a partial image around the central position provided in this application. Figure 8 illustrates the first headlight position information, including 2D frames corresponding to the paired headlights of oncoming vehicles. The center point of each 2D frame can be determined as the central position. Two central positions are determined in Figure 8.

[0082] a2. Determine the average quality value based on the image quality scores of the local images surrounding the center position.

[0083] In this application, the 2D frames corresponding to the paired headlights of the oncoming vehicle can be placed on the segmented image of the surrounding environment of the vehicle to determine the local image around the center position, obtain the image quality score corresponding to the local image around the center position, and determine the average quality value.

[0084] In this embodiment, the local images around the center position may include multiple local images around the center position, or may only include the local image where the center position is located.

[0085] In one possible implementation, the number of local images to be selected around the center position can be determined based on the number of local images obtained from the division, the position information of the first headlight (such as the size of the 2D frame), etc. For example, if the number of local images to be selected around the center position is determined to be 5×5, the average quality value is determined based on the image quality scores corresponding to the 5×5 local images around the center position; or if the number of local images to be selected around the center position is determined to be 3×3, the average quality value is determined based on the image quality scores corresponding to the 3×3 local images around the center position. It is understood that the above is only an exemplary description and should not be construed as a limitation on the embodiments of this application.

[0086] Please refer to Figure 8. The local images around the center position in Figure 8 include: four local images around one center position and a local image of another center position. The average quality can be determined based on these five local images.

[0087] In this embodiment of the application, the average quality value can be obtained by performing an arithmetic mean, geometric mean, or weighted mean on the image quality scores corresponding to the local images around the center position.

[0088] In this application example, a quality mean can be determined jointly based on the quality scores of multiple image quality scores corresponding to local images around two center positions, or two quality means can be determined separately based on the image quality scores corresponding to local images around each center position.

[0089] a3. Determine if the average mass value is less than the mass threshold to determine if oncoming vehicles are using high beams.

[0090] In this embodiment, if the average quality value is determined to be less than the quality threshold, it indicates that the image quality corresponding to the headlight area of ​​the oncoming vehicle in the surrounding environment image is poor, and the oncoming vehicle's lights affect the quality of the surrounding environment image captured by the vehicle. Therefore, it can be determined that the oncoming vehicle is using high beams. If the average quality value is determined to be greater than or equal to the quality threshold, it indicates that the image quality corresponding to the headlight area of ​​the oncoming vehicle in the surrounding environment image is good, and the oncoming vehicle's lights do not affect the quality of the surrounding environment image captured by the vehicle. Therefore, it can be determined that the oncoming vehicle is not using high beams.

[0091] It should be noted that in this embodiment, it is determined whether oncoming vehicles in each frame of the vehicle's surrounding environment image are using high beams. Each frame of the vehicle's surrounding environment image will determine a corresponding lighting usage, such as whether oncoming vehicles are using high beams or not.

[0092] The quality threshold in this embodiment can be set according to actual conditions, such as 0.5, 0.4, etc. It is understood that the quality threshold value is merely illustrative and should not be construed as a limitation on the embodiments of this application.

[0093] When two mass averages are determined, both mass averages must be less than a mass threshold before it can be determined that oncoming vehicles should use high beams.

[0094] The embodiments of this application can determine the average quality value corresponding to the headlight area of ​​the oncoming vehicle, and then compare it with the quality threshold to more accurately determine the lighting usage of the oncoming vehicle and output alarm information in a timely manner when the oncoming vehicle uses high beams.

[0095] S406. If it is determined that oncoming vehicles are using high beams in N consecutive frames of the surrounding environment image of the vehicle, output an alarm message.

[0096] Where N is a positive integer, and N is greater than 1. In the embodiments of this application, N can be 2, 3, or 4, etc. This ensures that an oncoming vehicle is using high beams in multiple consecutive frames of the vehicle's surrounding environment before an alarm message is output. This avoids the situation where a brief misjudgment of an oncoming vehicle using high beams due to external environmental interference leads to incorrect alarm message output, thus enabling more accurate alarm message output.

[0097] It is understood that the "output alarm information" in this embodiment is the same as the "output alarm information" in the above embodiment S201, so it will not be described again.

[0098] S407. Issue voice reminders and / or display text reminders, and / or identify oncoming vehicles using high beams on the vehicle's interface, and / or vibrate the steering wheel.

[0099] In this embodiment, voice reminders such as "Please be aware of road conditions, lighting conditions are poor" or "Lighting conditions are poor" can be issued through the vehicle's speakers; text reminders such as "Please be aware of road conditions, lighting conditions are poor" or "Lighting conditions are poor" can also be displayed through the vehicle's intelligent cockpit human-machine interface (HMI); oncoming vehicles using high beams can be highlighted on the HMI with bright colors such as yellow or red; and the steering wheel can be vibrated. This allows for multiple ways to prompt the user to take over, further ensuring driving safety. The HMI may include a central control screen, instrument panel, etc.

[0100] S408, control the high beam and low beam of the vehicle to flash alternately.

[0101] In one possible implementation, the embodiments of this application can use High Beam Assist (HMA) to control the high beam and low beam to flash alternately twice, politely reminding the driver of the oncoming vehicle to turn off the high beam, thereby further ensuring driving safety.

[0102] S409, Control the vehicle to slow down and drive on the right.

[0103] In this embodiment, if the vehicle's camera fails and the driver cannot take over in time, the intelligent driving system can adjust the control strategy in a timely manner, controlling the vehicle to slow down and move to the right, thus further ensuring driving safety.

[0104] In one possible implementation, if an oncoming vehicle is within a set distance range to the left of the vehicle, the vehicle is controlled to decelerate and move to the right; if an oncoming vehicle is outside the set distance range to the left of the vehicle, the vehicle is controlled to decelerate. The set distance can be X meters, and the value of X can be set according to the actual situation.

[0105] In this embodiment, the horn can also be used to remind drivers of oncoming vehicles to turn off their high beams, further ensuring driving safety.

[0106] It should be noted that, in the embodiments of this application, a scenario in which oncoming vehicles use high beams at night can be constructed to test whether the vehicle can recognize the use of high beams by oncoming vehicles at night, whether it can make special marks oncoming vehicles using high beams, whether it can flash the high beams twice alternately to remind the other driver, etc.; and whether the vehicle does not take corresponding actions after the forward-view camera is blocked.

[0107] As can be seen, the embodiments of this application can distinguish between the illumination of ordinary low beam headlights and the glare of high beam headlights through a perception network, promptly identify camera malfunction scenarios caused by oncoming vehicles using high beams, and specially mark oncoming vehicles using high beams so that the driver of the own vehicle can distinguish between oncoming vehicles using high beams and those not using high beams. It can also remind the driver of the own vehicle to take over and remind the driver of oncoming vehicles to turn off their high beams, achieving simultaneous reminders to both the driver of the own vehicle and the drivers of oncoming vehicles, thereby avoiding impacts on intelligent driving safety. Furthermore, the vehicle warning method provided in the embodiments of this application does not rely on LiDAR sensors to identify and obtain oncoming vehicle information, and can be applied to vision-based autonomous driving; it does not rely on data verification based on vehicle information preset on the server, and can identify whether more vehicles are using high beams; in assisted driving mode, it can remind the user of functional limitations and pay attention to driving safety; thus, it enhances the capabilities of the intelligent driving system and better ensures driving safety.

[0108] Please refer to Figure 9, which is a flowchart illustrating a training method for a perception network provided in this application. Figure 9 provides a more detailed explanation of the perception network used in the vehicle alarm method, based on the vehicle alarm methods provided in Figures 2 and 4. The vehicle alarm method provided in this application embodiment may further include a training method for the perception network, which may include the following steps:

[0109] S901. Input the image of the surrounding environment of the historical vehicle into the first training network to obtain the position information of the second headlight.

[0110] The historical vehicle surrounding environment images used in this application can originate from real-world environmental data collected in practice, such as visual image video streams. These visual image video streams can be acquired through front-view cameras, side-view cameras, rear-view cameras, fisheye cameras, etc. The historical vehicle surrounding environment images can include images collected under different orientations and weather conditions. These images may include oncoming vehicles using high beams, low beams, or no lights, or they may not include oncoming vehicles. This allows for the training of both the first and second training networks using a rich variety of historical vehicle surrounding environment images, resulting in a more accurate perception network.

[0111] The first training network in this application can be a Perspective View (PV) network, also known as a visual detection network, which can be used for dynamic target detection, such as 2D target detection, 3D target detection, attribute state estimation, etc. In this application, the PV network trained using historical images of the vehicle's surrounding environment can detect target vehicles around the vehicle and determine the headlight position information of the target vehicles.

[0112] Understandably, if the historical image of the vehicle's surrounding environment does not include oncoming vehicles, the second headlight position information can be a null value or a representation value, etc.

[0113] S902. Input the historical vehicle's surrounding environment image and the second headlight position information into the second training network to obtain the high beam status information.

[0114] Among them, the high beam status information is used to characterize whether oncoming vehicles in the historical vehicle's surrounding environment image are using high beams. The high beam status information can be any of the following: oncoming vehicles are using high beams, or oncoming vehicles are not using high beams.

[0115] The second training network in this application can be a camera failure network. The camera failure network can divide the historical images of the surrounding environment of the vehicle to obtain multiple training local images. Based on the second headlight position information and the image quality scores corresponding to the training local images, the high beam status information can be obtained.

[0116] It is understood that the internal execution operations of the second training network are consistent with the internal execution operations of the second sub-network in the above embodiment, so they will not be described again.

[0117] In the embodiments of this application, the first training network and the second training network can be neural networks. Specifically, the neural network can be an attention-based neural network, a convolutional neural network, a fully connected neural network, or other types of neural networks, etc. The specific form of the neural network can be determined according to the actual application scenario.

[0118] S903. Using the second headlight position information and the headlight position label information, determine the first loss value.

[0119] In this embodiment of the application, the second headlight position information may include multiple coordinate points corresponding to the paired headlights of the oncoming vehicle, a 2D bounding box corresponding to the paired headlights of the oncoming vehicle, etc. Correspondingly, the headlight position annotation information includes multiple annotated coordinate points corresponding to the paired headlights of the oncoming vehicle, annotated 2D bounding boxes corresponding to the paired headlights of the oncoming vehicle, etc.

[0120] S904. Determine the second loss value using the high beam status information and high beam status label information.

[0121] The high beam status label information in this application embodiment can be any of the following: oncoming vehicles are using high beams, oncoming vehicles are not using high beams.

[0122] S905. The first training network and the second training network are iteratively trained using the first loss value and the second loss value to obtain a perceptual network including the first sub-network and the second sub-network.

[0123] It is understandable that the first trained network is the first subnetwork, and the second trained network is the second subnetwork.

[0124] As can be seen, in the method provided by the embodiments of this application, a perception network including a first sub-network and a second sub-network is trained by using historical images of the environment surrounding the vehicle. In practical applications, the trained perception network can be used to determine the lighting usage of oncoming vehicles and to issue timely warnings for glare caused by the high beams of oncoming vehicles, thereby ensuring driving safety.

[0125] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0126] To facilitate better implementation of the above-described solutions in the embodiments of this application, related apparatus for implementing the above-described solutions is also provided below.

[0127] Please refer to Figure 10. An embodiment of this application provides a vehicle warning device 1000, which may include:

[0128] Module 1001 acquires images of the vehicle's surrounding environment;

[0129] The alarm module 1002 is used to output alarm information if the perception network determines that an oncoming vehicle is using high beams based on the image of the vehicle's surrounding environment; the perception network is used to detect the lighting status of oncoming vehicles based on the image of the vehicle's surrounding environment.

[0130] As can be seen, in this application, the sensor network is used to identify and detect the images of the vehicle's surrounding environment. When it is determined that an oncoming vehicle is using high beams, the alarm module will output alarm information in a timely manner. In this way, the glare caused by the high beams of oncoming vehicles can be warned in a timely manner, so that the driver can take over the vehicle in time, or the intelligent driving system can adjust the driving strategy in time, thereby reducing the impact of camera failure on vehicle driving and ensuring driving safety.

[0131] In some embodiments of this application, the perception network in the vehicle warning device 1000 includes: a first sub-network and a second sub-network. The first sub-network is used to detect the headlight position of oncoming vehicles based on the image of the surrounding environment of the vehicle. The second sub-network is used to detect the lighting usage of oncoming vehicles based on the detection results of the first sub-network and the image of the surrounding environment of the vehicle.

[0132] The alarm module 1002 includes: a detection unit, used to input the image of the vehicle's surrounding environment into the first sub-network for target detection, and obtain the position information of the first headlight corresponding to the oncoming vehicle in the image of the vehicle's surrounding environment;

[0133] The light determination unit is used to input the image of the vehicle's surrounding environment and the position information of the first headlight into the second sub-network to determine whether oncoming vehicles are using high beams.

[0134] In some embodiments of this application, the light determination unit in the vehicle warning device 1000 includes:

[0135] The input sub-unit is used to input the image of the vehicle's surrounding environment and the position information of the first headlight into the second sub-network;

[0136] The sub-unit is used to divide the image of the vehicle's surrounding environment through the second sub-network, thereby obtaining multiple local images corresponding to the image of the vehicle's surrounding environment.

[0137] A determination subunit is used to determine whether oncoming vehicles are using high beams based on the first headlight position information and the image quality score corresponding to the local image.

[0138] In some embodiments of this application, the determining subunit in the vehicle warning device 1000 is specifically used to determine the center position based on the first headlight position information; determine the average quality value based on the image quality scores corresponding to the local images around the center position; and determine that the average quality value is less than a quality threshold to determine that the oncoming vehicle is using high beams.

[0139] In some embodiments of this application, the vehicle alarm device 1000 displays multiple frames of images of the vehicle's surrounding environment, and the alarm module 1002 includes:

[0140] The alarm unit is used to output alarm information if the perception network determines that oncoming vehicles are using high beams in N consecutive frames of the vehicle's surrounding environment based on multiple frames of images of the vehicle's surrounding environment, where N is a positive integer and N is greater than 1.

[0141] In some embodiments of this application, the vehicle alarm device 1000 further includes: a training module;

[0142] The training module includes:

[0143] The input unit is used to input historical images of the vehicle's surrounding environment into the first training network to obtain the second headlight position information.

[0144] The input unit is also used to input the historical vehicle surrounding environment image and the second headlight position information into the second training network to obtain high beam status information. The high beam status information is used to characterize whether oncoming vehicles in the historical vehicle surrounding environment image are using high beams.

[0145] The loss value determination unit is used to determine the first loss value using the second headlight position information and the headlight position marking information;

[0146] The loss value determination unit is also used to determine a second loss value using the high beam status information and the high beam status label information;

[0147] The iterative training unit is used to iteratively train the first training network and the second training network using the first loss value and the second loss value to obtain a perceptual network including the first sub-network and the second sub-network.

[0148] In some embodiments of this application, the vehicle alarm device 1000 further includes:

[0149] The issuing module is used to issue voice reminders; and / or,

[0150] The display module is used to display text reminders; and / or,

[0151] The identification module is used to identify oncoming vehicles using high beams on the vehicle's interface; and / or,

[0152] Vibration module for vibrating the steering wheel;

[0153] The control module is used to control the alternating flashing of the vehicle's high beam and low beam headlights.

[0154] In some embodiments of this application, the control module in the vehicle alarm device 1000 is also used to control the vehicle to decelerate and move to the right.

[0155] It should be noted that the information interaction and execution process between the modules / units of the above-mentioned device are based on the same concept as the method embodiments of this application, and the resulting technical effects are the same as those of the method embodiments of this application. For details, please refer to the description in the method embodiments shown above in this application, and will not be repeated here.

[0156] This application also provides a computer storage medium, wherein the computer storage medium includes instructions that, when executed on a computer, cause the computer to perform some or all of the steps described in the above method embodiments.

[0157] The following describes a vehicle alarm device provided in an embodiment of this application. Please refer to Figure 11, which is a structural schematic diagram of a vehicle alarm device provided in an embodiment of this application. Specifically, the vehicle alarm device 1100 includes: a receiver 1101, a transmitter 1102, a processor 1103, a memory 1104 (wherein the number of processors 1103 in the vehicle alarm device 1100 can be one or more; Figure 11 shows one processor as an example), and an antenna. The processor 1103 may include an application processor 11031 and a communication processor 11032. In some embodiments of this application, the receiver 1101, transmitter 1102, processor 1103, and memory 1104 can be connected via a bus or other means.

[0158] Memory 1104 may include read-only memory and random access memory, and provides instructions and data to processor 1103. A portion of memory 1104 may also include non-volatile random access memory (NVRAM). Memory 1104 stores processor and operation instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0159] Processor 1103 controls the operation of the device. In specific applications, the various components of the device are coupled together through a bus system, which may include not only the data bus, but also power buses, control buses, and status signal buses. However, for clarity, all buses in the diagram are referred to as the bus system.

[0160] The vehicle alarm method disclosed in the above embodiments of this application can be applied to or implemented by the processor 1103. The processor 1103 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuits in the hardware of the processor 1103 or by instructions in software form. The processor 1103 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and may further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1103 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 1104. Processor 1103 reads the information from memory 1104 and, in conjunction with its hardware, completes the steps of the aforementioned vehicle alarm method.

[0161] Receiver 1101 can be used to receive input digital or character information, and to generate signal inputs related to device settings and function control. Transmitter 1102 can be used to output digital or character information through the first interface; transmitter 1102 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; transmitter 1102 may also include a display device such as a display screen.

[0162] In this embodiment, processor 1103 is used to execute the vehicle alarm method in the embodiments corresponding to Figures 2, 4, or 9. It should be noted that the specific manner in which the application processor 11031 in processor 1103 executes the aforementioned steps is based on the same concept as the method embodiments corresponding to Figures 1 to 9 in this application, and the resulting technical effects are the same as those in the method embodiments corresponding to Figures 1 to 9 in this application. For details, please refer to the descriptions in the method embodiments shown above in this application; they will not be repeated here.

[0163] This application embodiment also provides a vehicle 1200. Please refer to Figure 12, which is a structural schematic diagram of a vehicle provided in this application embodiment. The vehicle 1200 is configured for fully or partially automated driving mode. For example, the vehicle 1200 can control itself while in automated driving mode, and can determine the current state of the vehicle and its surrounding environment through human operation, determine the possible behavior of at least one other vehicle in the surrounding environment, and determine the confidence level corresponding to the probability of other vehicles performing possible behaviors, and control the vehicle 1200 based on the determined information. When the vehicle 1200 is in automated driving mode, the vehicle 1200 can also be set to operate without human interaction.

[0164] Vehicle 1200 may include various subsystems, such as a mobility system 1202, a sensor system 1204, a control system 1206, one or more peripheral devices 1208, a power supply 1210, a computer system 1212, and a user interface 1216. Optionally, vehicle 1200 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 1200 may be interconnected via wired or wireless means.

[0165] The mobility system 1202 may include components that provide powered motion to the vehicle 1200. In one embodiment, the mobility system 1202 may include an engine 1218, an energy source 1219, a transmission 1220, and wheels / tires 1221.

[0166] Engine 1218 can be an internal combustion engine, an electric motor, an air-compressed engine, or other combinations of engines, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air-compressed engine. Engine 1218 converts energy source 1219 into mechanical energy. Examples of energy source 1219 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 1219 can also provide energy to other systems of vehicle 1200. Transmission 1220 transmits mechanical power from engine 1218 to wheels 1221. Transmission 1220 may include a gearbox, a differential, and a drive shaft. In one embodiment, transmission 1220 may also include other components, such as a clutch. The drive shaft may include one or more axles that can be coupled to one or more wheels 1221.

[0167] Sensor system 1204 may include several sensors for sensing information about the environment surrounding vehicle 1200. For example, sensor system 1204 may include a positioning system 1222 (which may be a GPS system, a BeiDou system, or another positioning system), an inertial measurement unit (IMU) 1224, a radar 1226, a laser rangefinder 1228, and a camera 1230. Sensor system 1204 may also include sensors for the internal systems of the monitored vehicle 1200 (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensing data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a key function for the safe operation of autonomous vehicle 1200.

[0168] The positioning system 1222 can be used to estimate the geographical location of the vehicle 1200. An IMU 1224 is used to sense changes in the position and orientation of the vehicle 1200 based on inertial acceleration. In one embodiment, the IMU 1224 can be a combination of an accelerometer and a gyroscope. A radar 1226 can use radio signals to sense objects in the surrounding environment of the vehicle 1200, specifically millimeter-wave radar or lidar. In some embodiments, in addition to sensing objects, the radar 1226 can also be used to sense the speed and / or direction of travel of objects. A laser rangefinder 1228 can use lasers to sense objects in the environment in which the vehicle 1200 is located. In some embodiments, the laser rangefinder 1228 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components. A camera 1230 can be used to capture multiple images of the surrounding environment of the vehicle 1200. The camera 1230 can be a still camera or a video camera.

[0169] The control system 1206 controls the operation of the vehicle 1200 and its components. The control system 1206 may include various components, including a steering system 1232, a throttle 1234, a braking unit 1236, a computer vision system 1240, a trajectory control system 1242, and an obstacle avoidance system 1244.

[0170] The steering system 1232 is operable to adjust the forward direction of the vehicle 1200. For example, in one embodiment, it may be a steering wheel system. The throttle 1234 controls the operating speed of the engine 1218 and thus the speed of the vehicle 1200. The braking unit 1236 controls the deceleration of the vehicle 1200. The braking unit 1236 may use friction to slow down the wheels 1221. In other embodiments, the braking unit 1236 may convert the kinetic energy of the wheels 1221 into electrical current. The braking unit 1236 may also take other forms to slow down the rotational speed of the wheels 1221 to control the speed of the vehicle 1200. The computer vision system 1240 is operable to process and analyze images captured by the camera 1230 to identify objects and / or features in the environment surrounding the vehicle 1200. Objects and / or features may include traffic signals, road boundaries, and obstacles. The computer vision system 1240 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 1240 can be used to map the environment, track objects, estimate object speeds, etc. The route control system 1242 is used to determine the driving route and speed of the vehicle 1200. In some embodiments, the route control system 1242 may include a lateral planning module 12421 and a longitudinal planning module 12422, which are respectively used to combine data from the obstacle avoidance system 1244, GPS 1222, and one or more predetermined maps to determine the driving route and speed for the vehicle 1200. The obstacle avoidance system 1244 is used to identify, evaluate, and avoid or otherwise traverse obstacles in the environment of the vehicle 1200, which may specifically be physical obstacles and virtual moving objects that may collide with the vehicle 1200. In one example, the control system 1206 may add or replace components other than those shown and described. Alternatively, some of the components shown above may be reduced.

[0171] Vehicle 1200 interacts with external sensors, other vehicles, other computer systems, or users via peripheral device 1208. Peripheral device 1208 may include wireless communication system 1246, on-board computer 1248, microphone 1250, and / or speaker 1252. In some embodiments, peripheral device 1208 provides a means for a user of vehicle 1200 to interact with user interface 1216. For example, on-board computer 1248 may provide information to a user of vehicle 1200. User interface 1216 may also operate on-board computer 1248 to receive user input. On-board computer 1248 may be operated via a touchscreen. In other cases, peripheral device 1208 may provide a means for vehicle 1200 to communicate with other devices located within the vehicle. For example, microphone 1250 may receive audio (e.g., voice commands or other audio input) from a user of vehicle 1200. Similarly, speaker 1252 may output audio to a user of vehicle 1200. Wireless communication system 1246 may wirelessly communicate with one or more devices, either directly or via a communication network. For example, the wireless communication system 1246 may use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE, or 5G cellular communication. The wireless communication system 1246 may utilize a wireless local area network (WLAN) for communication. In some embodiments, the wireless communication system 1246 may utilize an infrared link, Bluetooth, or ZigBee to communicate directly with devices. Other wireless protocols, such as various vehicle communication systems, may also be used. For example, the wireless communication system 1246 may include one or more dedicated short-range communications (DSRC) devices that may enable public and / or private data communication between the vehicle and / or a roadside station.

[0172] Power source 1210 can provide power to various components of vehicle 1200. In one embodiment, power source 1210 can be a rechargeable lithium-ion or lead-acid battery. One or more such battery packs can be configured to provide power to various components of vehicle 1200. In some embodiments, power source 1210 and energy source 1219 can be implemented together, as is the case in some fully electric vehicles.

[0173] Some or all of the functions of vehicle 1200 are controlled by computer system 1212. Computer system 1212 may include at least one processor 1213, which executes instructions 1215 stored in a non-transitory computer-readable medium such as memory 1214. Computer system 1212 may also be multiple computing devices that control individual components or subsystems of vehicle 1200 in a distributed manner. Processor 1213 may be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, processor 1213 may be a dedicated device such as an application-specific integrated circuit (ASIC) or other hardware-based processor. Although FIG12 functionally illustrates the processor, memory, and other components of computer system 1212 in the same block, those skilled in the art will understand that the processor or memory may actually include multiple processors or memories not stored in the same physical housing. For example, memory 1212 may be a hard disk drive or other storage media located in a housing different from that of computer system 1212. Therefore, references to processor 1213 or memory 1214 will be understood as references to a collection of processors or memories that may or may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as the steering assembly and deceleration assembly, may each have their own processor that performs only calculations relevant to the component's specific function.

[0174] In all aspects described herein, processor 1213 may be located remotely from vehicle 1200 and may communicate wirelessly with vehicle 1200. In other aspects, some of the processes described herein are executed on processor 1213 located within vehicle 1200 while others are executed by remote processor 1213, including taking the necessary steps to perform a single operation.

[0175] In some embodiments, memory 1212 may contain instructions 1215 (e.g., program logic) that can be executed by processor 1213 to perform various functions of vehicle 1200, including those described above. Memory 1214 may also contain additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the mobility system 1202, sensor system 1204, control system 1206, and peripheral devices 1208. In addition to instructions 1215, memory 1214 may also store data such as road maps, route information, vehicle position, direction, speed, and other such vehicle data, as well as other information. This information may be used by vehicle 1200 and computer system 1212 during operation of vehicle 1200 in autonomous, semi-autonomous, and / or manual modes. User interface 1216 is used to provide or receive information from users of vehicle 1200. Optionally, the user interface 1216 may include one or more input / output devices within the set of peripheral devices 1208, such as a wireless communication system 1246, an in-vehicle computer 1248, a microphone 1250, and a speaker 1252.

[0176] Computer system 1212 can control the functions of vehicle 1200 based on inputs received from various subsystems (e.g., driving system 1202, sensor system 1204, and control system 1206) and from user interface 1216. For example, computer system 1212 can utilize inputs from control system 1206 to control steering system 1232 to avoid obstacles detected by sensor system 1204 and obstacle avoidance system 1244. In some embodiments, computer system 1212 is operable to provide control over many aspects of vehicle 1200 and its subsystems.

[0177] Alternatively, one or more of these components may be installed separately from or associated with the vehicle 1200. For example, the memory 1214 may exist partially or completely separately from the vehicle 1200. The components may be communicatively coupled together in a wired and / or wireless manner.

[0178] Optionally, the above components are merely examples. In practical applications, components in each of the above modules may be added or removed as needed. Figure 12 should not be construed as a limitation on the embodiments of this application. A vehicle traveling on a road, such as vehicle 1200 above, can identify objects in its surrounding environment to determine adjustments to its current speed. These objects can be other vehicles, traffic control equipment, or other types of objects. In some examples, each identified object can be considered independently, and based on the object's individual characteristics, such as its current speed, acceleration, and distance from the vehicle, the speed adjustment to be made by the vehicle can be determined.

[0179] Optionally, the vehicle 1200 or the computing devices associated with it, such as the computer system 1212, computer vision system 1240, and memory 1214 in Figure 12, can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of the others, so all identified objects can be considered together to predict the behavior of a single identified object. The vehicle 1200 can adjust its speed based on the predicted behavior of the identified objects. In other words, the vehicle 1200 can determine what steady state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered in determining the speed of the vehicle 1200, such as the lateral position of the vehicle 1200 in the road, the curvature of the road, the proximity of static and dynamic objects, etc. In addition to providing instructions to adjust the vehicle's speed, the computing device can also provide instructions to modify the steering angle of the vehicle 1200 so that the vehicle 1200 follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the vehicle 1200 (e.g., cars in adjacent lanes on the road).

[0180] The aforementioned vehicle 1200 can be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, and train, etc., and this application embodiment does not impose any special limitations.

[0181] In this embodiment, the processor 1213 in the vehicle 1200 is used to execute the vehicle alarm method in the embodiments corresponding to FIG2, FIG4, or FIG9. It should be noted that the specific manner in which the processor 1213 executes the aforementioned steps is based on the same concept as the method embodiments corresponding to FIG1 to FIG9 in this application, and the resulting technical effects are the same as those in the method embodiments corresponding to FIG1 to FIG9 in this application. For details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0182] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the vehicle alarm method described in the embodiments shown in Figures 2, 4, or 9.

[0183] This application also provides a computer program product, which includes a program that, when run on a computer, causes the computer to perform the vehicle alarm method described in the embodiments shown in FIG2, FIG4 or FIG9 above.

[0184] This application provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform the vehicle alarm method described in the embodiments shown in Figures 2, 4, or 9 above.

[0185] This application also provides a chip system including a processor for supporting the device in implementing the functions involved in the vehicle alarm method described in the embodiments shown in Figures 2, 4, or 9, such as sending or processing data and / or information involved in the vehicle alarm method described in the embodiments shown in Figures 2, 4, or 9. In one possible design, the chip system also includes a memory for storing necessary program instructions and data of the device. The chip system may be composed of chips or may include chips and other discrete devices.

[0186] This application also provides a chip, including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of the electronic device and send signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to perform the vehicle alarm method described in the embodiments shown in FIG2, FIG4 or FIG9 above.

[0187] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of a program in the first aspect of the method.

[0188] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the accompanying drawings of the device embodiments provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0189] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, and of course, it can also be implemented by special hardware including application-specific integrated circuits, special-purpose CLUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0190] In the above embodiments, the implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product.

[0191] The aforementioned computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

Claims

1. A vehicle alarm method, characterized in that, The method includes: Acquire images of the vehicle's surrounding environment; If the perception network determines that an oncoming vehicle is using high beams based on the image of the vehicle's surrounding environment, it outputs a warning message; the perception network is used to detect the oncoming vehicle's lighting usage based on the image of the vehicle's surrounding environment.

2. The method according to claim 1, characterized in that, The perception network includes a first sub-network and a second sub-network. The first sub-network is used to detect the headlight position of the oncoming vehicle based on the image of the surrounding environment of the vehicle. The second sub-network is used to detect the lighting usage of the oncoming vehicle based on the detection result of the first sub-network and the image of the surrounding environment of the vehicle. The determination that an oncoming vehicle is using high beams includes: The image of the vehicle's surrounding environment is input into the first sub-network for target detection, thereby obtaining the position information of the first headlights of the oncoming vehicle in the image of the vehicle's surrounding environment. The image of the vehicle's surrounding environment and the position information of the first headlight are input into the second sub-network to determine that the oncoming vehicle is using high beams.

3. The method according to claim 2, characterized in that, The step of inputting the image of the vehicle's surrounding environment and the position information of the first headlight into the second sub-network to determine that the oncoming vehicle is using high beams includes: The image of the vehicle's surrounding environment and the position information of the first headlight are input into the second sub-network; The vehicle's surrounding environment image is divided by the second sub-network to obtain multiple local images corresponding to the vehicle's surrounding environment image; Based on the first headlight position information and the image quality score corresponding to the local image, it is determined that the oncoming vehicle is using high beams.

4. The method according to claim 3, characterized in that, The step of determining that the oncoming vehicle is using high beams based on the first headlight position information and the image quality score corresponding to the local image includes: The center position is determined based on the first headlight position information; The average quality value is determined based on the image quality scores of the local images surrounding the central location. Determining that the average mass value is less than a mass threshold is used to determine that the oncoming vehicle is using high beams.

5. The method according to any one of claims 1 to 4, characterized in that, The image of the vehicle's surrounding environment is composed of multiple frames, and the output alarm information includes: If the perception network determines, based on multiple frames of the vehicle's surrounding environment images, that an oncoming vehicle is using high beams in N consecutive frames of the vehicle's surrounding environment images, and outputs the warning information, where N is a positive integer and is greater than 1.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: The image of the surrounding environment of the historical vehicle is input into the first training network to obtain the position information of the second headlight; The image of the surrounding environment of the historical vehicle and the position information of the second headlight are input into the second training network to obtain the high beam status information. The high beam status information is used to characterize whether oncoming vehicles in the image of the surrounding environment of the historical vehicle are using high beams. The first loss value is determined using the second headlight position information and the headlight position marking information; The second loss value is determined using the high beam status information and high beam status label information; The first training network and the second training network are iteratively trained using the first loss value and the second loss value to obtain the perception network including the first sub-network and the second sub-network.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Issue voice reminders and / or display text reminders, and / or identify oncoming vehicles using high beams on the vehicle's interface, and / or vibrate the steering wheel; Control the high beam and low beam of the vehicle to flash alternately.

8. The method according to any one of claims 1 to 7, characterized in that, The alarm information is used to alert the vehicle's camera to malfunction, and the method further includes: Control the vehicle to slow down and move to the right.

9. A vehicle alarm device, characterized in that, The device includes: The acquisition module acquires images of the vehicle's surrounding environment. The alarm module is used to output alarm information if the perception network determines that an oncoming vehicle is using high beams based on the image of the vehicle's surrounding environment; the perception network is used to detect the oncoming vehicle's lighting usage based on the image of the vehicle's surrounding environment.

10. A vehicle alarm device, comprising at least one processor, the processor being configured to invoke computer instructions in a memory to cause the vehicle alarm device to perform the vehicle alarm method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the vehicle alarm method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the vehicle alarm method as described in any one of claims 1 to 8.

13. A chip, characterized in that, The device includes one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from the memory of the electronic device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device performs the vehicle alarm method as described in any one of claims 1 to 8.

14. A vehicle, characterized in that, It includes at least one processor, the processor being configured to invoke computer instructions in memory to cause the vehicle alarm device to perform the vehicle alarm method as described in any one of claims 1 to 8.

Images