Vehicle light recognition method, storage medium and intelligent device
The vehicle image is detected and matched through the preset headlight key point detection network, and the vehicle lights are selected using the characteristics of the headlight group setting, and the vehicle existence is verified, which solves the problem of misidentification of ambient lights and improves the The accuracy and safety of automatic driving’s headlight recognition.
Patent Information
- Application Number
- CN202311790560.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-25
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-12-25
AI Technical Summary
Existing car light recognition methods are prone to misidentify ambient lights as car lights, which affects the safety and reliability of autonomous driving. Especially when the ambient lights are unstable light sources, it may lead to misidentification of frequent flickering of the car lights.
The preset headlight key point detection network is used to detect and match the image. The candidate headlights belonging to the same set of headlights preset on the vehicle are filtered out through the characteristics of the headlight group setting, and their positions are judged and the target is output. The position of the headlights, check whether there is a vehicle at the target headlights to improve identification accuracy.
Through screening and verification, the accuracy of car light recognition is improved, the misidentification of ambient lights is reduced, and the safety and reliability of autonomous driving is enhanced.
Smart Images

Figure CN117456511B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of autonomous driving technology, and in particular to a vehicle light recognition method, storage medium and intelligent device. Background Art
[0002] When controlling the vehicle's autonomous driving, it is usually necessary to accurately identify the status of other vehicle lights in the surrounding environment, and control the vehicle's autonomous driving based on the status of other vehicle lights, so as to improve the safety and reliability of autonomous driving. In addition to other vehicle lights, there may also be various ambient lights such as decorative lights and street lights in the environment, so it is necessary to accurately identify which are the lights, and then determine the vehicle's light status based on these lights.
[0003] At present, the conventional method of headlight recognition is mainly to detect the key points of the headlights, and then convert the key points of the headlights to the world coordinate system to obtain the headlight recognition results. However, this method can easily misidentify the above-mentioned ambient lights as headlights, thereby affecting the safety and reliability of autonomous driving. In addition, ambient lights are usually unstable light sources. If the ambient lights are mistakenly identified as headlights, then based on the light of these misidentified ambient lights, it may be recognized that the headlights are in a state of frequent flashing, which will not only affect the safety and reliability of autonomous driving, but also cause trouble to the occupants and reduce the car experience of the occupants.
[0004] Accordingly, the art needs a new technical solution to solve the above problems. Summary of the invention
[0005] In order to overcome the above-mentioned defects, the present application is proposed to provide a vehicle headlight recognition method, a storage medium and an intelligent device that solve or at least partially solve the technical problem of how to accurately recognize vehicle heads.
[0006] In a first aspect, a vehicle light recognition method is provided, the method comprising:
[0007] Performing vehicle headlight detection on the image to obtain candidate vehicle headlights in the image;
[0008] Matching the candidate headlights to obtain candidate headlights that belong to the same group of headlights preset on the vehicle;
[0009] Taking the candidate headlights belonging to the same group of headlights preset on the vehicle as target headlights;
[0010] The position of the target vehicle light is obtained and outputted.
[0011] In a technical solution of the above vehicle light recognition method, the step of performing vehicle light detection on an image to obtain candidate vehicle lights in the image includes:
[0012] Using a preset headlight key point detection network to perform headlight key point detection on the image to obtain headlight key points in the image;
[0013] The key points of the headlights are used as the candidate headlights.
[0014] In a technical solution of the above-mentioned vehicle headlight recognition method, matching the candidate vehicle headlights to obtain candidate vehicle headlights belonging to the same set of vehicle headlights preset on the vehicle includes:
[0015] For each of the headlight key points, using the preset headlight key point detection network, predicting the positions of other headlight key points that belong to the same group of headlights preset on the vehicle as the headlight key point;
[0016] Obtain other key points of vehicle lights at the position;
[0017] The headlight key point and other headlight key points at the position are taken as candidate headlights belonging to the same group of headlights preset on the vehicle.
[0018] In a technical solution of the above-mentioned vehicle headlight recognition method, matching the candidate vehicle headlights to obtain candidate vehicle headlights belonging to the same set of vehicle headlights preset on the vehicle includes:
[0019] For each of the headlight key points, using the preset headlight key point detection network, obtain the feature similarity between the headlight key point and each of the other headlight key points;
[0020] According to the feature similarity, other headlight key points that belong to the same group of headlights preset on the vehicle as the headlight key point are obtained;
[0021] The headlight key point and the other headlight key points obtained according to the feature similarity are used as candidate headlights belonging to the same group of headlights preset on the vehicle.
[0022] In a technical solution of the above-mentioned vehicle light recognition method, the preset vehicle light key point detection network is obtained by:
[0023] Obtain image samples and their key point annotation data;
[0024] Using the image samples and the headlight key point annotation data thereof, training the preset headlight key point detection network;
[0025] The headlight key point annotation data includes the position and light group information of each headlight key point in the image sample, and the light group information of the headlight key point includes information of other headlight key points that belong to the same group of headlights preset on the vehicle as the headlight key point.
[0026] In a technical solution of the above vehicle light recognition method, the step of obtaining and outputting the position of the target vehicle light includes:
[0027] Obtaining the position of the target vehicle light and determining whether there is a vehicle at the position;
[0028] If yes, output the position of the target headlight;
[0029] If not, the position of the target vehicle light is not output.
[0030] In a technical solution of the above vehicle light recognition method, the image is an image captured by a vehicle-mounted camera, and before determining whether there is a vehicle at the position, the method further includes:
[0031] According to the position of the target headlight, determining whether the target headlight is a close-range target relative to the vehicle equipped with the vehicle-mounted camera;
[0032] If yes, continue to determine whether there is a vehicle at the location;
[0033] If not, the position of the target vehicle light is directly outputted without determining whether there is a vehicle at the position.
[0034] In a technical solution of the above vehicle light recognition method, the step of determining whether the target vehicle light is a close-range target relative to the vehicle equipped with the vehicle-mounted camera comprises:
[0035] Obtaining the distance between the target headlight and the vehicle equipped with the vehicle-mounted camera;
[0036] Determining whether the distance is less than or equal to a set value;
[0037] If yes, then the target headlight is the close-range target;
[0038] If not, the target vehicle light is not the close-range target.
[0039] In a second aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored in the computer-readable storage medium, wherein the program codes are suitable for being loaded and run by a processor to execute the method described in any one of the technical solutions of the above-mentioned vehicle light recognition method.
[0040] In a third aspect, a smart device is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any one of the technical solutions of the above-mentioned vehicle light recognition method is implemented.
[0041] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:
[0042] In a technical solution for implementing the headlight recognition method provided by the present application, after performing headlight detection on an image to obtain candidate headlights, the candidate headlights can be matched to obtain candidate headlights belonging to the same group of headlights preset on the vehicle, and these candidate headlights belonging to the same group of headlights preset on the vehicle are used as target headlights, and the positions of these target headlights are obtained and output. The headlights on a vehicle are usually arranged in groups. For example, two high beams and two low beams can be arranged on the front of the vehicle, and the two high beams belong to the same group of headlights, and the two low beams belong to the same group of headlights. By obtaining candidate headlights belonging to the same group of headlights preset on the vehicle as target headlights, the above-mentioned characteristics of the headlights being arranged in groups are utilized to screen the candidate headlights, filter out the candidate headlights that may actually be ambient lights, and ensure the accuracy of headlight recognition.
[0043] In another technical solution of the headlight recognition method provided by the present application, after obtaining the target headlight, it is also possible to determine whether there is a vehicle at the position of the target headlight; if so, it indicates that the possibility that the target headlight is an ambient light is relatively low, so the position of the target headlight can be output; if not, it indicates that the possibility that the target headlight is an ambient light is relatively high, and in order to ensure the accuracy of headlight recognition, it needs to be filtered out, so the position of the target headlight is no longer output. Based on the above implementation scheme, the accuracy of the target headlight can be verified using the vehicle detection results to further improve the accuracy of headlight recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The disclosure of the present application will become easier to understand with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. Among them:
[0045] Figure 1 is a schematic flow chart of the main steps of a vehicle light recognition method according to an embodiment of the present application;
[0046] Figure 2 is a flowchart of main steps of a method for acquiring candidate vehicle lights by performing vehicle light detection on an image according to an embodiment of the present application;
[0047] Figure 3 It is a flowchart of main steps of a method for obtaining candidate headlights belonging to the same group of headlights preset on a vehicle according to an embodiment of the present application;
[0048] Figure 4 is a flowchart of main steps of a method for obtaining candidate headlights belonging to the same group of headlights preset on a vehicle according to another embodiment of the present application;
[0049] Figure 5 is a flowchart of main steps of a method for acquiring and outputting a target vehicle light position according to an embodiment of the present application;
[0050] Figure 6 This is a flow chart of main steps of a method for acquiring a target headlight based on a headlight key point and outputting the position of the target headlight according to an embodiment of the present application;
[0051] Figure 7 is an image containing both vehicle lights and ambient lights in a night scene according to an embodiment of the present application;
[0052] Figure 8 yes Figure 7 Schematic diagram of the key points of the headlights in the image shown;
[0053] Fig. 9 It is a schematic diagram of the main structure of a smart device according to an embodiment of the present application.
[0054] List of reference numerals:
[0055] 11: memory; 12: processor. DETAILED DESCRIPTION
[0056] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.
[0057] In the description of this application, "processor" may include hardware, software or a combination of the two. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Computer-readable storage media include any suitable medium that can store program code, such as a disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc.
[0058] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.
[0059] The user personal information processed by this application will vary depending on the specific product / service scenario, and shall be based on the specific scenario in which the user uses the product / service, and may involve the user's account information, device information, driving information, vehicle information or other related information. This application will treat the user's personal information and its processing with a high degree of diligence.
[0060] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.
[0061] The following is a description of an embodiment of the vehicle light recognition method provided in the present application.
[0062] See attached Figure 1 , Figure 1 FIG. 1 is a flow chart of the main steps of a vehicle light recognition method according to an embodiment of the present application. Figure 1 As shown, the vehicle light recognition method in the embodiment of the present application mainly includes the following steps S101 to S104.
[0063] Step S101: performing vehicle headlight detection on an image to obtain candidate vehicle headlights in the image.
[0064] Specifically, by performing vehicle headlight detection on an image, vehicle headlights existing in the image can be obtained, and these vehicle headlights are used as candidate vehicle headlights.
[0065] In some embodiments, when performing vehicle light detection on an image, it is possible to detect whether a light point or a light area in the image is a vehicle light. For example, the image may be an image taken by a camera of a highway at night, and the image includes light points or light areas generated by light sources such as vehicle lights and ambient lights. Vehicle light detection may be used to detect which of these light points or light areas are vehicle lights. It should be noted that in this embodiment, a conventional vehicle light detection method may be used to detect whether a light point or a light area in an image is a vehicle light, and this embodiment does not specifically limit this method.
[0066] Step S102: matching candidate headlights to obtain candidate headlights that belong to the same group of headlights preset on the vehicle.
[0067] Different types of headlights are usually arranged in groups in different areas of the vehicle, so the candidate headlights belonging to the same group of headlights preset on the vehicle are actually headlights of the same type arranged in groups on the vehicle, wherein each group of headlights arranged in groups includes at least two headlights.
[0068] For example, a low beam is provided on each side of the front of the vehicle, and the two low beams form a group of lights in the front area. A high beam is also provided on each side of the front of the vehicle, and the two high beams also form a group of lights in the front area. The group of lights formed by the low beams and the group of lights formed by the high beams are two different groups of lights. When candidate lights belonging to the same group of lights preset on the vehicle are obtained, these candidate lights either belong to the group of lights formed by the low beams or the group of lights formed by the high beams, and do not belong to both groups of lights at the same time.
[0069] Step S103: taking candidate headlights belonging to the same group of headlights preset on the vehicle as target headlights.
[0070] For those candidate lights that cannot belong to the same group of lights preset on the vehicle together with other candidate lights, these candidate lights are most likely to be ambient lights and need to be filtered or discarded. Therefore, in this step, the candidate lights that belong to the same group of lights preset on the vehicle can be directly used as target lights to continue the light position recognition.
[0071] Step S104: Acquire and output the position of the target vehicle light.
[0072] The position of the target headlight includes but is not limited to the position of the target headlight in the image, the position in the world coordinate system, the position in the coordinate system of the camera that captured the image, etc. Those skilled in the art can flexibly obtain the position of the target headlight in different coordinate systems according to actual needs, and this embodiment does not specifically limit this.
[0073] Based on the method described in steps S101 to S104 above, the characteristic of the headlights being arranged in groups can be utilized to screen the candidate headlights and filter out the candidate headlights that may actually be ambient lights, thereby ensuring the accuracy of headlight recognition.
[0074] In an application scenario according to an embodiment of the present application, when controlling a vehicle to drive automatically on a road at night, in addition to the current vehicle (hereinafter referred to as the self-vehicle), there are other normally traveling vehicles on the road, and there are some ambient lights around the road. In order to ensure the safety of autonomous driving, it is necessary to accurately identify the status of the headlights of other vehicles. To this end, the environmental image around the self-vehicle can be collected by a vehicle-mounted camera, and then the headlight recognition method according to the embodiment of the present application can be used to perform headlight recognition on the environmental image to obtain the position of the headlights of other vehicles, and then the headlight status is analyzed based on the image information at the position of the headlights, such as flashing of the headlights. Finally, the vehicle control driving is controlled according to the headlight status, such as braking, detouring the vehicle in front, etc.
[0075] The above steps S101, S102 and S104 are further explained below.
[0076] 1. Explain step S101.
[0077] In some implementations of the above step S101, Figure 2 In the following steps S1011 to S1012, vehicle headlight detection is performed on the image to obtain candidate vehicle headlights.
[0078] Step S1011: using a preset headlight key point detection network to perform headlight key point detection on the image to obtain the headlight key points in the image.
[0079] By using a preset headlight key point detection network, a headlight key point can be obtained for each headlight in the image, that is, a headlight key point represents a headlight.
[0080] In this embodiment, a conventional network in the field of neural network technology can be used to construct a preset headlight key point detection network. This embodiment does not specifically limit the structure of the network, as long as it can be used to detect the headlight key points of an image and obtain the headlight key points.
[0081] Step S1012: taking the key points of the headlights as candidate headlights.
[0082] Based on the method described in steps S1011 to S1012 above, the candidate vehicle lights can be obtained quickly and accurately by using neural network and key point detection technology.
[0083] In some implementations of the present application, a preset vehicle headlight key point detection network can be obtained through the following steps 11 to 12.
[0084] Step 11: Obtain image samples and their headlight key point annotation data.
[0085] The key point annotation data of the headlights includes the position and light group information of each key point of the headlights in the image sample, and the light group information of the key point of the headlights includes the information of other key points of the headlights that belong to the same group of headlights preset on the vehicle as the key point of the headlights. The information of the other key points of the headlights may be the position of the other key points of the headlights. Based on the light group information of the key points of the headlights, it is possible to accurately obtain which key points of the headlights belong to the same group of headlights as the key point of the headlights.
[0086] Step 12: Use the image samples and their headlight key point annotation data to train a preset headlight key point detection network.
[0087] Based on the position of the headlight key points, the trained headlight key point detection network can accurately detect the position of the headlight key points in the image, and based on the light group information, the trained headlight key point detection network can detect the headlight key points and which headlight key points belong to the same group of headlights preset on the vehicle after obtaining the position of the headlight key points.
[0088] Based on the methods described in steps 11 to 12 above, a headlight key point detection network can be trained that has the ability to detect the positions of headlight key points and detect which headlight key points belong to the same group of headlights preset on the vehicle. In this way, a headlight key point detection network can be used to detect headlights in an image to obtain candidate headlights and match the candidate headlights, thereby screening out candidate headlights that belong to the same group of headlights preset on the vehicle.
[0089] It should be noted that in this embodiment, conventional network training methods in the field of machine learning technology can be used to train the headlight key point detection network using image samples and their headlight key point annotation data. This embodiment does not specifically limit the training method, as long as the training can be completed and the trained network has the above two capabilities at the same time.
[0090] 2. Explain step S102.
[0091] In some implementations of the above step S102, Figure 3 The following steps S201 to S203 are shown to match the candidate headlights to obtain candidate headlights that belong to the same group of headlights preset on the vehicle.
[0092] Step S201: For each headlight key point, a preset headlight key point detection network is used to predict the positions of other headlight key points that belong to the same group of headlights preset on the vehicle as the headlight key point. The preset headlight key point detection network can be trained using the method described in steps 11 to 12 in the above embodiment.
[0093] Step S202: Acquire other key points of the vehicle headlights at the position.
[0094] If there are other headlight key points at this position, it means that the current headlight key point and the other headlight key point belong to the same group of headlights preset on the vehicle.
[0095] If there are no other headlight key points at this position, it means that the current headlight key point cannot form a group with other headlight key points. The current headlight key point is very likely to be the key point of the ambient light. At this time, the current headlight key point can be directly filtered or discarded, and no longer look for other headlight key points that belong to the same group of headlights preset on the vehicle.
[0096] Step S203: taking the key point of the headlight and other key points of the headlight at the position as candidate headlights belonging to the same group of headlights preset on the vehicle.
[0097] Based on the method described in steps S201 to S203 above, the candidate lights can be accurately matched by first predicting the position (i.e., the positions of other headlight key points that belong to the same group of headlights preset on the vehicle as the headlight key point) and then acquiring the headlight key points based on the position.
[0098] In some implementations of the above step S102, Figure 4 The following steps S301 to S303 are shown to match the candidate headlights to obtain the candidate headlights belonging to the same group of headlights preset on the vehicle.
[0099] Step S301: for each headlight key point, a preset headlight key point detection network is used to obtain the feature similarity between the headlight key point and each other headlight key point.
[0100] Specifically, when a preset headlight key point detection network is used to obtain each headlight key point in the image, the feature of each headlight key point can be obtained, and then the similarity between the feature of the headlight key point and the feature of each other headlight key point, that is, the feature similarity, can be obtained.
[0101] Step S302: based on the feature similarity, other headlight key points that belong to the same group of headlights preset on the vehicle as the headlight key point are obtained.
[0102] Different types of headlights are usually set in groups in different areas of the vehicle. The higher the feature similarity between two headlight key points, the more likely it is that the two headlight key points are located in the same area of the vehicle and the more similar the types of the headlights are. In other words, the higher the possibility that the two headlight key points belong to the same group of headlights preset on the vehicle. Therefore, other headlight key points that belong to the same group of headlights preset on the vehicle can be screened based on feature similarity.
[0103] In this embodiment, a similarity threshold can be set in advance. If the feature similarity between the key points of two headlights is greater than the similarity threshold, it can be determined that the two belong to the same set of headlights preset on the vehicle; if the feature similarity between the key points of two headlights is less than or equal to the similarity threshold, the two do not belong to the same set of headlights preset on the vehicle.
[0104] In this embodiment, if the feature similarity between the current headlight key point and each of the other headlight key points is less than or equal to the similarity threshold, it indicates that the current headlight key point is very likely to be the key point of the ambient light. At this time, the current headlight key point can be directly filtered or discarded, and no other headlight key points belonging to the same group of headlights preset on the vehicle are searched. Those skilled in the art can flexibly set the value of the similarity threshold according to actual needs, and this embodiment does not specifically limit this.
[0105] Step S303: taking the key point of the headlight and other key points of the headlight obtained according to the feature similarity as candidate headlights belonging to the same group of headlights preset on the vehicle.
[0106] Based on the method described in steps S301 to S303 above, the features of the key points of the headlights obtained by the preset key point detection network of the headlights can be reused to quickly and accurately complete the matching of the candidate headlights.
[0107] 3. Explain step S104.
[0108] In some implementations of the above step S104, Figure 5 The following steps S1041 to S1044 are shown to obtain and output the position of the target vehicle light.
[0109] Step S1041: Acquire the position of the target vehicle headlight.
[0110] The position of the target headlight includes but is not limited to the position of the target headlight in the image, the position in the world coordinate system, the position in the coordinate system of the camera that captured the image, etc. In some preferred embodiments, the position of the target headlight in the world coordinate system can be obtained.
[0111] Step S1042: Determine whether there is a vehicle at the location.
[0112] The vehicle lights are usually set on the outside of the vehicle body, such as the front or rear of the vehicle. If the target vehicle light is not an ambient light but a vehicle light, then there should be a vehicle near the target vehicle light. Therefore, if there is a vehicle at this position, it means that the possibility that the target vehicle light is an ambient light is extremely low, and the position of the target vehicle light can be output, so the process goes to step S1043;
[0113] If there is no vehicle at the position, it indicates that the target headlight is likely to be an ambient light. If the position of the target headlight is output, the accuracy of headlight recognition will be reduced. Therefore, the position of the target headlight may not be output, and the process goes to step S1044.
[0114] Step S1043: Output the position of the target vehicle light.
[0115] Step S1044: The position of the target vehicle light is not output.
[0116] Based on the methods described in steps S1041 to S1044 above, the accuracy of the target headlights may be verified again by determining whether there are vehicles near the target headlights.
[0117] Furthermore, in some implementations of the above-mentioned step S104, the image may be an image captured by a vehicle-mounted camera. Before determining whether there is a vehicle at the position (i.e., before executing step S1042), it is also possible to determine whether to directly output the position of the target vehicle lights through the following steps 21 to 22 without continuing to execute steps S1042 to S1044.
[0118] Step 21: According to the position of the target headlight, determine whether the target headlight is a close target relative to the vehicle equipped with the vehicle-mounted camera.
[0119] If it is a close-range target, it means that the vehicle where the target headlight is located (hereinafter referred to as the target vehicle) is relatively close to the vehicle equipped with the vehicle-mounted camera (hereinafter referred to as the camera vehicle), which has a relatively large impact on the safe driving of the camera vehicle. It is necessary to further verify whether the target headlight is accurate, so go to step 22;
[0120] If it is not a close-range target, it means that the target vehicle is far away from the camera vehicle and has little impact on the safe driving of the camera vehicle. In order to save computing power, it is no longer necessary to verify whether the target headlights are accurate, and thus go to step 23.
[0121] In some embodiments, whether the target headlight is a close-range target can be determined in the following manner: obtaining the distance between the target headlight and the vehicle equipped with the vehicle-mounted camera, and determining whether the distance is less than or equal to a set value; if so, the target headlight is a close-range target; if not, the target headlight is not a close-range target. It should be noted that those skilled in the art can flexibly set the value of the above-mentioned set value according to actual needs, and this embodiment does not specifically limit this.
[0122] Step 22: Continue to determine whether there is a vehicle at the location, and output the location of the target vehicle light based on the determination result, that is, continue to execute steps S1042 to S1044.
[0123] Step 23: Directly output the position of the target vehicle light, and no longer determine whether there is a vehicle at the position, that is, no longer execute steps S1042 to S1044
[0124] Based on the methods described in steps 21 to 23 above, the accuracy of the target headlights can be selectively verified based on whether the target headlights are close targets relative to the vehicle equipped with the vehicle-mounted camera, which can effectively balance the needs of improving the accuracy of headlight recognition and saving computing power.
[0125] The following is combined with Figures 6 to 8 , the embodiment of the vehicle light recognition method provided in the present application is briefly described again.
[0126] See attached Figure 6 , Figure 6 FIG. 2 is a schematic flow chart of the main steps of the vehicle light recognition method in this embodiment. Figure 6 As shown, the vehicle light recognition method mainly includes the following steps S401 to S407.
[0127] Step S401: Detect key points of vehicle lights in the image.
[0128] In this step, the method described in steps S1011 to S1012 in the above-mentioned embodiment can be used to detect the key points of the vehicle lights.
[0129] Step S402: for each headlight key point, determine whether a matching point of the headlight key point can be found, wherein the matching point refers to other headlight key points that belong to the same group of headlights preset on the vehicle as the headlight key point.
[0130] In this step, the method described in step S102 of the above embodiment can be used to obtain the matching point of each headlight key point. For each headlight key point, if the matching point of the headlight key point can be found, the lane key point and its matching point are used as the target headlight, and step S403 is continued; if it cannot be found, it indicates that the headlight key point is the key point of the ambient light, and the process goes to step S407 to discard the headlight key point.
[0131] See attached Figure 7 and attached Figure 8 , Figure 7 The following example shows an image containing both car lights and ambient lights at night. After detecting the key points of the car lights in this image, we can get Figure 8 Lane key points 1-4 are shown. Lane key points 1 and 2 are matched points to each other, and lane key points 3 and 4 cannot find matching points and can be discarded.
[0132] Step S403: Obtain the position of the target vehicle headlight in the world coordinate system.
[0133] Step S404: Determine whether the target vehicle headlight is a close-range target.
[0134] In this step, the method described in steps 21 to 22 in the above embodiment may be used to determine whether the target vehicle headlight is a close-range target.
[0135] If the target vehicle headlight is a close-range target, go to step S405;
[0136] If the target vehicle headlight is not a close-range target, the process proceeds to step S406.
[0137] Step S405: Determine whether there is a vehicle near the target headlight; if there is a vehicle, go to step S406; if there is no vehicle, go to step S407.
[0138] Step S406: Output the position of the target vehicle light.
[0139] Step S407: discarding the key points of the headlights.
[0140] Based on the method described in steps S401 to S407 above, the position information of the vehicle lights can be accurately identified from the image.
[0141] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order, and they can be executed simultaneously (in parallel) or in other orders. These adjusted schemes are equivalent to the technical schemes described in this application, and therefore will also fall within the scope of protection of this application.
[0142] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium that can carry the computer program code. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.
[0143] Another aspect of the present application also provides a computer-readable storage medium.
[0144] In an embodiment of a computer-readable storage medium according to the present application, the computer-readable storage medium may be configured to store a program for executing the vehicle light recognition method of the above method embodiment, and the program may be loaded and run by a processor to implement the above vehicle light recognition method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-transitory computer-readable storage medium.
[0145] Another aspect of the present application also provides a smart device.
[0146] In an embodiment of an intelligent device according to the present application, the intelligent device may include at least one processor; and a memory connected to the at least one processor in communication; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any of the above embodiments is implemented. The intelligent device described in the present application may include a driving device, a smart car, a robot, and the like. Fig. 9 , Fig. 9 FIG. 4 exemplarily shows that the memory 11 and the processor 12 are communicatively connected via a bus.
[0147] In some embodiments of the present application, the smart device may further include at least one sensor for sensing information. The sensor is communicatively connected to any type of processor mentioned in the present application. Optionally, the smart device may further include an autonomous driving system for guiding the smart device to drive itself or assist in driving. The processor communicates with the sensor and / or the autonomous driving system to complete the method described in any of the above embodiments.
[0148] So far, the technical solution of the present application has been described in conjunction with an embodiment shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.
Claims
1. A vehicle light recognition method, characterized in that: The method comprises: Using a preset headlight key point detection network to perform headlight key point detection on the image to obtain headlight key points in the image, and using the headlight key points as candidate headlights; Matching the candidate lights to obtain candidate lights that belong to the same set of lights preset on the vehicle, including: for each of the key points of the lights, using the preset key point detection network of the lights, predicting the positions of other key points of the lights that belong to the same set of lights preset on the vehicle as the key point of the lights; obtaining other key points of the lights at the positions; using the key point of the lights and other key points of the lights at the positions as candidate lights that belong to the same set of lights preset on the vehicle; wherein, if there are no other key points of the lights at the positions, discarding the candidate lights; Taking the candidate headlights belonging to the same group of headlights preset on the vehicle as target headlights; Acquire and output the position of the target vehicle light; The image is an image captured by a vehicle-mounted camera; the method further comprises: judging whether the target vehicle light is a close-range target relative to the vehicle equipped with the vehicle-mounted camera according to the position of the target vehicle light; if the target vehicle light is the close-range target and there is no vehicle at the position of the target vehicle light, discarding the target vehicle light; The preset headlight key point detection network is obtained by: acquiring image samples and headlight key point annotation data thereof; using the image samples and headlight key point annotation data thereof to train the preset headlight key point detection network; wherein the headlight key point annotation data includes the position and headlight group information of each headlight key point in the image sample, and the headlight group information of the headlight key point includes information of other headlight key points that belong to the same group of headlights preset on the vehicle as the headlight key point.
2. The method according to claim 1, characterized in that The matching of the candidate vehicle lights to obtain candidate vehicle lights belonging to the same group of vehicle lights preset on the vehicle includes: For each of the headlight key points, using the preset headlight key point detection network, obtain the feature similarity between the headlight key point and each of the other headlight key points; According to the feature similarity, other headlight key points that belong to the same group of headlights preset on the vehicle as the headlight key point are obtained; The headlight key point and the other headlight key points obtained according to the feature similarity are used as candidate headlights belonging to the same group of headlights preset on the vehicle.
3. The method according to claim 1, characterized in that The determining whether the target vehicle light is a close-range target relative to the vehicle equipped with the vehicle-mounted camera comprises: Obtaining the distance between the target headlight and the vehicle equipped with the vehicle-mounted camera; Determining whether the distance is less than or equal to a set value; If yes, then the target headlight is the close-range target; If not, the target vehicle light is not the close-range target.
4. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the vehicle light recognition method according to any one of claims 1 to 3.
5. A smart device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores a computer program, and when the computer program is executed by the at least one processor, the vehicle light recognition method according to any one of claims 1 to 3 is implemented.
