Signal light detection equipment, vehicle control method and system

CN119013713APending Publication Date: 2024-11-22YINWANG INTELLIGENT TECHNOLOGIES CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202280094787.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2022-04-15
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

It is difficult for self-driving vehicles to accurately recognize traffic light instructions in environments with insufficient lighting (such as at night, rainy days, and snowy days), resulting in inaccurate driving control.

Method used

Design a signal light detection device that uses a filter component to transmit the signal light emitted by the signal light, and converts it into light information of the recognition result through the photodetector, and combines it with the environmental image information obtained by the camera to control vehicle driving.

Benefits of technology

It improves the accuracy of signal light recognition and reliability of driving control under insufficient lighting conditions, reduces recognition errors and system overhead, and improves data processing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119013713A_ABST
    Figure CN119013713A_ABST
Patent Text Reader

Abstract

The invention discloses signal light detection equipment, a vehicle control method and a system. The signal light detection equipment (110) comprises a light filtering assembly (111) and a photoelectric detector (112); the light filtering assembly (111) is used for transmitting signal light emitted by a signal lamp in a target area; the photoelectric detector (112) is used for converting the signal light into first light information and outputting the first light information, and the first light information is used for indicating an identification result of the signal light emitted by the signal lamp in the target area. Signal light emitted by the signal lamp is filtered through the light filtering assembly (111), the signal light is converted into first light information indicating the identification result of the signal lamp, and accurate identification of information indicated by the signal lamp is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Signal light detection device, vehicle control method and system Technical Field

[0001] The present application relates to the field of intelligent vehicle technology, and in particular to a signal light detection device, a vehicle control method and a system. Background Art

[0002] With the increasing development of autonomous vehicle technology, the requirements for autonomous vehicles' traffic capabilities are becoming increasingly stringent. Currently, autonomous vehicles rely primarily on machine vision to identify traffic lights, controlling whether to park or drive according to the traffic light instructions. However, due to the limited accuracy of machine vision in color recognition, in some scenarios (such as low light at night, or in rain, snow, or fog), the dim light from traffic lights makes it difficult to accurately identify traffic light instructions.

[0003] Summary of the Invention

[0004] The embodiments of the present application provide a signal light detection device, a vehicle control method, and a system, in order to accurately identify the information indicated by traffic lights.

[0005] In a first aspect, an embodiment of the present application provides a signal light detection device, comprising: a filter component and a photodetector; the filter component is used to transmit the signal light emitted by a signal lamp; the photodetector is used to convert the signal light into first light information and output the first light information, wherein the first light information is used to indicate the recognition result of the signal light emitted by the signal lamp.

[0006] The signal light detection device provided in the first aspect filters the signal light emitted by the signal light through the filter component, and converts the signal light into first light information indicating the recognition result of the signal light, thereby achieving accurate recognition of the information indicated by the signal light.

[0007] In combination with the first aspect, in a possible implementation manner of the first aspect, the filter assembly includes a plurality of filters, each of which is used to transmit signal light of different wavelengths, and the signal light emits signal light of one wavelength at the same time.

[0008] The signal light detection device provided by this embodiment can transmit signal light of different wavelengths emitted by the signal lamp through multiple filters in the filter assembly, simplifying the structure of the signal light detection device and making the signal light detection device suitable for detecting signal light of different wavelengths.

[0009] In combination with the first aspect and a possible implementation of the above-mentioned first aspect, in another possible implementation of the first aspect, the method also includes: the device also includes: an optical fiber jumper; the optical fiber jumper is used to establish an optical transmission channel between each filter in the filter assembly and the photodetector; the optical fiber jumper transmits the signal light output by the filter assembly to the photodetector through the corresponding optical transmission channel.

[0010] The signal light detection device provided by this embodiment can reduce the loss during light transmission.

[0011] In combination with the first aspect and the above possible implementation manners of the first aspect, in another possible implementation manner of the first aspect, the filter assembly is disposed on the inner side of a windshield of the vehicle.

[0012] The signal light detection device provided in this embodiment enables the filter component to be aligned with the signal light, thereby facilitating the reception of the signal light emitted by the signal light.

[0013] In a second aspect, an embodiment of the present application provides a vehicle control method, comprising: obtaining first light information from a signal light detection device, the first light information being used to indicate a recognition result of a signal light emitted by a signal lamp in a target area; synchronously obtaining an image of the target area from a camera; identifying the image of the target area to obtain environmental information of the target area; and controlling vehicle driving according to the environmental information and the first light information shown.

[0014] Through the method provided in the second aspect, the processor can obtain the recognition result of the signal light emitted by the signal light in the target area from the signal light detection device, that is, the first light information, and then combine the first light information and the environmental information based on image recognition to control the vehicle's driving, thereby improving the reliability of driving control.

[0015] In combination with the second aspect, in a possible implementation of the second aspect, the vehicle driving control is performed based on the environmental information and the first light information, including: when the environmental information meets a preset condition, generating a driving instruction based on the first light information; and controlling the vehicle driving according to the driving instruction.

[0016] Through the method provided by this embodiment, when the processor determines that the environmental information meets the preset conditions, it chooses to control the vehicle driving according to the first light information, thereby avoiding the situation where the first light information cannot accurately reflect the information indicated by the traffic light, and also controlling the vehicle driving based on the first light information, thereby improving the reliability of driving control.

[0017] In combination with the second aspect and a possible implementation of the above-mentioned second aspect, in another possible implementation of the second aspect, the preset condition includes: the environmental information indicates that the light intensity of the target area is lower than the light threshold; and / or, the environmental information indicates that the target area includes the signal light; and / or, the recognition accuracy of the second light information indicated by the environmental information is less than the accuracy threshold, and the second light information is the recognition result of the signal light emitted by the signal light obtained by recognizing the image of the target area.

[0018] Through the method provided by this embodiment, when the environmental information indicates that the light intensity of the target area is lower than the light threshold, the processor controls the vehicle driving based on the first light information, thereby avoiding recognition errors caused by recognizing the signal light emitted by the signal light based on image recognition technology when the light intensity is low, thereby improving recognition accuracy; when the environmental information indicates that the target area includes the signal light, the processor controls the vehicle driving based on the first light information, thereby avoiding interference with the signal light detection equipment caused by other lights on the road (such as street lights, etc.), thereby improving recognition accuracy; when the recognition accuracy of the second light information indicated by the environmental information is less than the accuracy threshold, the processor controls the vehicle driving based on the first light information, thereby avoiding recognizing the signal light emitted by the signal light based on the first light information when the second light information can accurately reflect the signal light emitted by the signal light, thereby reducing system overhead and improving data processing efficiency.

[0019] In combination with the second aspect and the possible implementations of the second aspect mentioned above, in another possible implementation of the second aspect, the vehicle driving control is performed based on the environmental information and the first light information, including: determining the third light information through weighted operation based on the first light information and the second light information in the environmental information, the second light information being the recognition result of the signal light emitted by the signal light obtained by recognizing the image of the target area; generating a driving instruction based on the third light information; and controlling the vehicle driving based on the driving instruction.

[0020] The method provided in this embodiment comprehensively considers the recognition results of the signal light emitted by the signal light reflected by the first light information and the second light information, and obtains the final third light information, thereby improving the recognition accuracy of the signal light emitted by the signal light.

[0021] Optionally, the driving instruction includes at least one of a driving instruction, a parking instruction, and a speed control instruction.

[0022] In combination with the second aspect and the possible implementations of the second aspect mentioned above, in another possible implementation of the second aspect, the image of the target area is identified to obtain environmental information of the target area, including: using a pre-trained visual model to identify the image of the target area to obtain second light information, wherein the second light information is a recognition result of the signal light emitted by the signal light obtained by identifying the image of the target area.

[0023] Through the method provided in this embodiment, the image of the target area can be accurately recognized based on the visual model to obtain the second light information.

[0024] In combination with the second aspect and the possible implementations of the second aspect mentioned above, in another possible implementation of the second aspect, the method also includes: obtaining multiple training samples, the training samples being images of signal lights transmitted through a filter, the filter being used to transmit the signal light emitted by the signal light; and training the initial network model based on the multiple training samples to obtain the visual model.

[0025] Through the method provided in this embodiment, the processor uses the image of the traffic light transmitted through the filter as a training sample to train the initial network model. The obtained visual model can combine the features of the filtered image to identify the signal light emitted by the traffic light in the target area, thereby improving the accuracy of signal light recognition.

[0026] In a third aspect, the present application provides a vehicle control method, comprising: transmitting signal light emitted by a signal lamp in a target area through a filtering component; performing photoelectric conversion on the signal light to obtain first light information, wherein the first light information is used to indicate an identification result of the signal light emitted by the signal lamp in the target area; and outputting the first light information.

[0027] In combination with the third aspect, in a possible implementation of the third aspect, the filter assembly includes multiple filters, which are respectively used to transmit signal lights of different wavelengths emitted by the signal lamp in the target area, and the signal lamp emits signal light of one wavelength at the same time.

[0028] The beneficial effects of the vehicle control method provided by the third aspect and the possible implementation methods of the third aspect can be referred to the beneficial effects brought about by the first aspect, the second aspect and the possible implementation methods, and will not be repeated here.

[0029] In a fourth aspect, an embodiment of the present application provides a device, comprising: an acquisition unit for acquiring first light information from a signal light detection device, wherein the first light information is used to indicate the recognition result of the signal light emitted by the signal light in the target area; the acquisition unit is also used to synchronously acquire an image of the target area from a camera; an identification unit for identifying the image of the target area and obtaining environmental information of the target area; and a control unit for controlling the driving of the vehicle based on the environmental information and the first light information.

[0030] In combination with the above-mentioned fourth aspect, in a possible implementation of the fourth aspect, the vehicle control unit is specifically used to: generate a driving instruction according to the first light information when the environmental information meets the preset conditions; and perform driving control of the vehicle according to the driving instruction.

[0031] In combination with the above-mentioned fourth aspect and a possible implementation of the above-mentioned fourth aspect, in another possible implementation of the fourth aspect, the preset conditions include: the environmental information indicates that the light intensity of the target area is lower than the light threshold; and / or, the environmental information indicates that the target area includes the signal light; and / or, the recognition accuracy of the second light information indicated by the environmental information is less than the accuracy threshold, and the second light information is the recognition result of the signal light emitted by the signal light obtained by recognizing the image of the target area.

[0032] In combination with the above-mentioned fourth aspect and the possible implementation methods of the above-mentioned fourth aspect, in another possible implementation method of the fourth aspect, the vehicle control unit is specifically used to: determine the third light information through weighted operation based on the first light information and the second light information in the environmental information, where the second light information is the recognition result of the signal light emitted by the signal light obtained by recognizing the image of the target area; generate a driving instruction based on the third light information; and control the vehicle's driving according to the driving instruction.

[0033] In combination with the above fourth aspect and each possible implementation manner of the above fourth aspect, in another possible implementation manner of the fourth aspect, the driving instruction includes: at least one of: a driving instruction, a parking instruction, and a speed control instruction.

[0034] In combination with the above-mentioned fourth aspect and each possible implementation of the above-mentioned fourth aspect, in another possible implementation of the fourth aspect, the recognition unit is specifically used to: recognize the image of the target area through a pre-trained visual model to obtain second light information, and the second light information is the recognition result of the signal light emitted by the signal light obtained by recognizing the image of the target area.

[0035] In combination with the above-mentioned fourth aspect and each possible implementation manner of the above-mentioned fourth aspect, in another possible implementation manner of the fourth aspect, the acquisition unit is further used to: acquire multiple training samples, the training samples are images of signal lights transmitted through a filter, and the filter is used to transmit the signal light emitted by the signal light; according to the multiple training samples, the initial network model is trained to obtain a visual model.

[0036] In combination with the fourth aspect and each possible implementation manner of the fourth aspect, in another possible implementation manner of the fourth aspect, the acquisition unit is further used to: acquire the first light information through a camera, and the camera is deployed with a filter component.

[0037] The beneficial effects of the device provided by the fourth aspect and each possible implementation method of the fourth aspect can be referred to the beneficial effects brought about by the first aspect, the second aspect and each possible implementation method, and will not be repeated here.

[0038] In a fifth aspect, an embodiment of the present application provides a chip, comprising: a processor for calling and running a computer program from a memory, so that a device equipped with the chip executes the method provided in the second aspect and various possible implementations of the second aspect.

[0039] In a sixth aspect, an embodiment of the present application provides a vehicle control system, comprising a signal light detection device as provided in the first aspect and each possible implementation manner of the first aspect, and a chip as provided in the fourth aspect.

[0040] In the seventh aspect, an embodiment of the present application provides a device comprising: a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory, and executing the method provided in the second aspect and various possible implementations of the second aspect.

[0041] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, which enables a computer to execute the method provided in the second aspect and various possible implementations of the second aspect.

[0042] In a ninth aspect, an embodiment of the present application provides a computer program product, comprising computer program instructions, which enable a computer to execute the method provided in the second aspect and various possible implementations of the second aspect.

[0043] In a tenth aspect, an embodiment of the present application provides a computer program, which enables a computer to execute the method provided in the second aspect and various possible implementations of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] FIG1 is a schematic diagram of a vehicle control system provided in an embodiment of the present application;

[0045] FIG2 is a schematic structural diagram of a signal light detection device provided in an embodiment of the present application;

[0046] FIG3 is a schematic diagram of a vehicle control process provided in an embodiment of the present application;

[0047] FIG4 is a flow chart of a vehicle control method provided in an embodiment of the present application;

[0048] FIG5 is a flow chart of another vehicle control method provided in an embodiment of the present application;

[0049] FIG6 is a schematic block diagram of a device provided in an embodiment of the present application;

[0050] FIG7 is a schematic structural diagram of a device according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0052] The technical solutions of the embodiments of the present application may be implemented by a mobile device or a device deployed in the mobile device. The mobile device may have any appearance, such as an intelligent vehicle, an intelligent robot, etc. The intelligent vehicle may be an autonomous vehicle that automatically controls all functions, or an assisted driving vehicle that automatically controls some functions to provide driving assistance.

[0053] The following description will be made using an example in which the mobile device is a smart vehicle.

[0054] Machine vision technology uses computers to simulate human vision, extracting information from images of objective objects, processing it, and understanding it for practical detection, measurement, and control. Using machine vision, intelligent vehicles can determine their driving paths while driving and control their movements based on the indications of traffic lights. These lights, for example, can be traffic lights that direct traffic, typically consisting of red, green, and yellow lights, including but not limited to motor vehicle lights, non-motor vehicle lights, pedestrian crossing lights, direction indicators (arrow lights), lane lights, flashing warning lights, and road and railway intersection lights. However, while machine vision has high accuracy for image recognition, it has lower accuracy for color and light recognition. In particular, in some special environments, such as at night, in rainy, snowy, and foggy weather, where light intensity is low, machine vision can struggle to accurately identify the information indicated by traffic lights.

[0055] In order to solve the above technical problems, the embodiment of the present application filters the signal light emitted by the signal light and converts the signal light into light information indicating the recognition result of the signal light (such as the first light information below), so as to accurately identify the information indicated by the signal light.

[0056] Figure 1 is a schematic diagram of a vehicle control system 100 provided in an embodiment of the present application. As shown in Figure 1 , the vehicle control system 100 includes at least a signal light detection device 110 and a processor 120. The signal light detection device 110 and the processor 120 are connected via a wired or wireless connection.

[0057] The signal light detection device 110 can be deployed on the inside of the vehicle's windshield. Of course, this application does not limit the deployment location of the signal light detection device 110. For example, it can also be deployed on the front of the vehicle (such as the vehicle bumper), on the outside of the vehicle's windshield, or any other location that is convenient for receiving the signal light of the signal light.

[0058] The signal light detection device 110 can be used to sense the signal light emitted by a traffic light. The traffic light can be located in a target area. The area within the detection range of the signal light detection device 110 is the target area, or a partial area within the detection range of the signal light detection device 110 is the target area. When the traffic light is a monochromatic light, the signal light emitted by the traffic light always has a single wavelength, for example, the wavelength of the signal light emitted by the traffic light belongs to the red wavelength band. When the traffic light is a multicolor light, the traffic light emits a single wavelength at the same time, for example, the wavelength of the signal light emitted by a red, green, and yellow traffic light at the current moment belongs to the red wavelength band. This application does not limit the color of the signal light emitted by the traffic light. For example, it can be red, green, yellow, blue, etc. Furthermore, the signal light detection device 110 can obtain first light information based on the signal light and output the first light information, for example, by sending the first light information to the processor 120 or outputting the first light information to a display for display, thereby indicating the recognition result of the traffic light in the target area through the first light information.

[0059] The processor 120 can be integrated with the above-mentioned signal light detection device 110 into a single device, or the processor 120 can be independently deployed in the vehicle, or the processor 120 can be deployed outside the vehicle, for example, in the cloud, which is not limited in this application. The processor 120 can be implemented as a chip. In some specific implementations, the processor 120 can be deployed in an on-board computing device, such as a mobile data center (MDC), an electronic control unit, a domain controller, etc.

[0060] The processor 120 can control the vehicle's driving based on the first light information sent by the light detection device 110. Alternatively, the processor 120 can control the vehicle's driving based on the first light information and environmental information sent by the light detection device 110. The specific implementation of the processor 120 is described in detail below. The environmental information can be obtained based on machine vision recognition. For example, the processor can synchronously acquire image information of a target area through a camera and recognize the image of the target area to obtain environmental information of the target area.

[0061] In some embodiments, the vehicle control system 100 further includes a camera 130, which can be connected to the processor 120 via a wired or wireless connection. The camera 130 can be deployed at the front of the vehicle, for example, it can be a front-facing camera among multiple cameras deployed around the vehicle based on machine vision technology. The camera 130 can capture images of a target area and send the captured images to the processor 120, which then recognizes the captured images, thereby implementing machine vision.

[0062] FIG2 is a schematic diagram of the structure of a signal light detection device provided in an embodiment of the present application. As shown in FIG2 , in some embodiments, the signal light detection device 110 may include a filter assembly 111 and a photodetector 112. The filter assembly is connected to the photodetector. The filter assembly 111 can be deployed inside the vehicle's windshield. Of course, this application does not limit the deployment location of the filter assembly; for example, it can also be deployed at the front of the vehicle (such as the vehicle bumper) or outside the vehicle's windshield, or anywhere else convenient for receiving signal light from a signal light. The photodetector 112 can be integrated with the filter assembly 111 or deployed in a distributed manner, which is not limited in this application. When the photodetector 112 is deployed in a distributed manner with the filter assembly 111, this application does not limit the deployment location of the photodetector 111; for example, it can be deployed inside the vehicle body or inside the rear end of the vehicle. When the photodetector 112 is deployed in a distributed manner with the filter assembly 111, the photodetector 111 can also be integrated with the processor 120.

[0063] The filter assembly 111 may include one or more filters. Generally speaking, the filters included in the filter assembly 111 correspond to the colors of the signal light emitted by the traffic light, and each filter is used to filter a type of signal light. For example, the traffic light to be identified may emit signal light of three colors: red, green, and yellow. The filter assembly 111 includes a filter 1111 for filtering red light, a filter 1112 for filtering green light, and a filter 1113 for filtering yellow light. When the traffic light in the target area emits red light, the filter assembly 111 may filter the signal light through the filter that filters red light. When the traffic light in the target area emits yellow light, the filter assembly 111 may filter the signal light through the filter that filters yellow light. When the traffic light in the target area emits green light, the filter assembly 111 may filter the signal light through the filter that filters green light. Of course, the number of filters included in the filter assembly 111 can be greater than the number of signal lights that can be emitted by signal lights in the target area, so that it can be applied to different signal light recognition scenarios without deploying a filter assembly for each signal light.

[0064] As shown in FIG2 , when the filter assembly 111 includes multiple filters, the filters (such as filters 1111 to 1113 ) can be arranged side by side without overlapping, so that different filters can filter light of corresponding wavelengths respectively.

[0065] For any of the above-mentioned filters (such as filters 1111 to 1113), its optical characteristic is to transmit light within a specific wavelength band and absorb light outside the specific wavelength band. This characteristic of the filter can be achieved by coating. In the embodiment of the present application, the wavelength band of light transmitted by the filter can be a wavelength band with the wavelength of the signal light emitted by the signal light as the center wavelength. For example, the wavelength of the red light emitted by the signal light is 680nm, so the filter 1111 used to filter red light in the filter assembly 111 can filter the wavelength band with 680nm as the center wavelength. In the embodiment of the present application, the wavelength range of light transmitted by the filter can be set to be smaller to improve the accuracy of the signal light detection device. For example, when driving at night, street lights, advertising lights, green lights, etc. will cause interference to the signal light detection device. In this case, the wavelength range of light transmitted by the filter should be set to be smaller, or the wavelength range of light transmitted by the filter can be set to be larger to make the signal light detection device more sensitive to the signal light.

[0066] The photodetector 112 can utilize the photoconductivity effect, a physical phenomenon in which radiation causes a change in the electrical conductivity of an irradiated material, to convert signal light into an electrical signal. Therefore, in the present application, the photodetector 112 can be used to convert signal light into an electrical signal representing the signal light (e.g., first optical information). This first optical information can be used to indicate the result of identifying the signal light emitted by a signal light. For example, the first optical information can indicate whether the signal light emitted by a signal light in a target area is red, green, or yellow. Exemplarily, the first optical information is a level signal whose magnitude is related to at least one of the power, wavelength, and half-maximum width of the signal light.

[0067] Furthermore, the photodetector 112 may output the first light information, for example, output the first light information to the processor 120 .

[0068] In some embodiments, the filter assembly 111 and the photodetector 112 can be connected via an optical fiber jumper 113. The optical fiber jumper 113 is used to establish an optical transmission channel between each filter (e.g., filters 1111 to 1113) in the filter assembly 111 and the photodetector 112. As shown in FIG2 , the red light emitted by the signal light in the target area is filtered by the red light filter 1111 and input to port 113-11 of the optical fiber jumper 113, and then input to the photodetector 112 via port 113-12. The green light emitted by the signal light in the target area is filtered by the green light filter 1112 and then input to port 113-21 of the optical fiber jumper 113, and then input to the photodetector 112 via port 113-22. The yellow light emitted by the signal light in the target area is filtered by the yellow light filter 1113 and then input to port 113-31 of the optical fiber jumper 113, and then input to the photodetector 112 via port 113-32.

[0069] The optical fiber jumper 113 is a medium for light transmission, and can transmit light therein with very little loss through a relatively short optical fiber.

[0070] As mentioned above, when the filter assembly 111 includes multiple filters, the signal light filtered by each filter is input to the photodetector 112 through a different fiber jumper port. The photodetector 112 can distinguish different signal lights based on the input port of the signal light.

[0071] As mentioned above, the processor 120 performs driving control of the vehicle based on the first light information in at least two possible implementations:

[0072] Method 1: The processor 120 can control the vehicle's driving based on the first light information sent by the light detection device 110. For example, when the first light information indicates that the signal light emitted by the signal light in the target area is red, the processor 120 can control the vehicle to park; when the first light information indicates that the signal light emitted by the signal light in the target area is green, the processor 120 can control the vehicle to drive; when the first light information indicates that the signal light emitted by the signal light in the target area is yellow, the processor 120 can control the vehicle to slow down, etc. Optionally, the processor 120 can generate a driving instruction based on the first light information and control the vehicle's driving based on the driving instruction. Corresponding to the example above, when the first light information indicates that the signal light emitted by the signal light in the target area is red, the driving instruction is a drive instruction; when the first light information indicates that the signal light emitted by the signal light in the target area is green, the driving instruction is a park instruction; when the first light information indicates that the signal light emitted by the signal light in the target area is yellow, the driving instruction is a speed control instruction, etc.

[0073] Method 2: The processor 120 may control the vehicle's driving based on the first light information and environmental information sent by the light detection device 110. There are several possible implementations for the processor 120 to control the vehicle's driving based on the first light information and environmental information:

[0074] In a first possible implementation of the second approach described above, the processor 120 can determine, based on the environmental information, whether to control vehicle driving based on the first light information. For example, when the environmental information meets a preset condition, the processor 120 controls vehicle driving based on the first light information; when the environmental information does not meet the preset condition, the processor 120 controls vehicle driving based on the second light information. The second light information is a recognition result of the signal light emitted by a traffic light in the target area, obtained by recognizing an image of the target area. For example, the second light information can be a recognition result of the signal light emitted by a traffic light in the target area based on machine vision.

[0075] The pre-conditions include but are not limited to at least one of the following:

[0076] Environmental information indicates that the light intensity of the target area is below a light threshold;

[0077] The environmental information indicates that the target area includes the signal light;

[0078] The recognition accuracy of the second light information indicated by the environmental information is less than the accuracy threshold.

[0079] It should be noted that regarding the above-mentioned preset conditions, when the environmental information indicates that the light intensity in the target area is lower than the light threshold, the signal light of the signal lamp is identified based on the first light information detected by the above-mentioned signal light detection device. Compared with machine vision, it can more accurately reflect the signal light of the signal lamp in the target area; because the signal light detection device cannot recognize the shape of the object, it may be interfered by other lights (such as street lights, advertising lights, etc.), and detect other lights as lights emitted by the signal lamp. Therefore, when the target area indicated by the environmental information includes a signal lamp, the processor 120 determines the signal light emitted by the signal lamp based on the first light information detected by the signal light detection device, which can improve the reliability of driving control; when the recognition accuracy of the second light information is less than the accuracy threshold, the accuracy of the recognition result based on machine vision is low, so the processor can more accurately reflect the signal light of the signal lamp in the target area based on the first light information detected by the above-mentioned signal light detection device.

[0080] Optionally, the second light information may be included in the environmental information.

[0081] Optionally, the processor 120 may input the dynamically captured image of the target area into a pre-trained visual model to recognize the image of the target area and obtain the second light information. The recognition accuracy of the second light information may be output by the visual model.

[0082] Optionally, the training of the visual model may include but is not limited to the following methods: the processor 120 obtains a plurality of training samples, each training sample is an image of a signal light transmitted through a filter, and the filter is used to transmit the signal light emitted by the signal light in the target area. For example, a filter component 131 can be deployed at the front end of the camera 130 in Figure 1. The filter component 131 can be the same as the filter component 111 in the signal light detection device 110 in Figure 2. The processor 120 can collect training samples through the camera 130 equipped with the filter component 131. Furthermore, the processor 120 trains the initial network model based on the collected multiple training samples, and ends the training process to obtain the visual model when the loss function converges to a preset value or the recognition accuracy of the model reaches a preset accuracy.

[0083] It should be noted that the multiple training samples can be pre-collected by other devices and are not limited to the sample collection process performed by the processor 120. It should also be understood that the training process of the above-mentioned visual model can also be performed by a server (such as a cloud server). After the server trains the visual model, the processor 120 can obtain the visual model from the server to recognize the image captured by the camera based on the visual model. Of course, the processor 120 can send the image captured by the camera to the server and receive the result returned by the server after recognizing the image based on the visual model.

[0084] Generally speaking, a visual model trained on images filtered by a light filter is primarily used to recognize images of a target area after the filter has been filtered, thereby improving the accuracy of model recognition. Of course, this application does not exclude the possibility that a visual model trained on images filtered by a disc can also be used to recognize images of a target area that has not been filtered by a light filter.

[0085] Referring to FIG3 , the processor 120 exemplarily determines whether the environmental information satisfies a preset condition. Upon determining that the environmental information satisfies the preset condition, the processor 120 activates a signal light detection device and detects the signal light through the signal light detection device. For example, when the first light information sent by the signal light detection device indicates that the signal light has changed from red to green, the processor 120 controls the vehicle to move. When the first light information sent by the signal light detection device indicates that the signal light has changed from green to red, the processor 120 controls the vehicle to park. Upon determining that the ambient mirror information does not satisfy the preset condition, the processor 120 controls the vehicle's driving based on machine vision.

[0086] In a second possible implementation of the above-mentioned method 2, the processor 120 determines whether to perform driving control of the vehicle in combination with the environmental information and the first light information based on the environmental information. Exemplarily, when the environmental information meets the preset conditions, the processor 120 determines the third light information through weighted operation based on the first light information and the second light information in the environmental information, and controls the driving of the vehicle based on the third light information. In the above-mentioned weighted operation process, the weights of the first light information and the second light information may be the same or different, and the weights of the first light information and the second light information may be related to the illumination intensity of the target area or the recognition accuracy of the second light information. For example, the lower the illumination intensity, the greater the weight of the first light information and the smaller the weight of the second light information; for another example, the higher the recognition accuracy of the second light information, the greater the weight of the second light information and the smaller the weight of the first light information. Exemplarily, when the environmental information does not meet the preset conditions, the processor 120 may control the driving of the vehicle based on the second light information.

[0087] For example, when the third light information indicates that the signal light in the target area emits red light, the processor 120 controls the vehicle to park; when the third light information indicates that the signal light in the target area emits green light, the processor 120 controls the vehicle to drive; when the third light information indicates that the signal light in the target area emits yellow light, the processor 120 controls the vehicle to slow down, etc.

[0088] Exemplarily, the processor 120 may generate a driving instruction based on the third light information and control the vehicle's driving according to the driving instruction. For example, when the third light information indicates that the signal light in the target area emits red light, the processor 120 generates a parking instruction and controls the vehicle to park according to the parking instruction; when the third light information indicates that the signal light in the target area emits green light, the processor 120 generates a driving instruction and controls the vehicle to drive according to the driving instruction; when the third light information indicates that the signal light in the target area emits yellow light, the processor 120 generates a speed control instruction and controls the vehicle to decelerate (or accelerate) according to the speed control instruction.

[0089] In the third possible implementation of the above-mentioned method 2, the processor 120 can combine the first light information and the second light information in the environmental information to control the driving of the vehicle. Exemplarily, the processor 120 combines the first light information and the second light information in the environmental information to control the driving of the vehicle, which is similar to the solution in the second possible implementation of the above-mentioned method 2. The difference from the second possible implementation of the above-mentioned method 2 is that in the third possible implementation, the processor 120 does not need to determine whether the environmental information meets the preset conditions, and directly controls the driving of the vehicle based on the first light information and the second light information. Compared with the second possible implementation, the third possible implementation can improve the processing efficiency of the processor 120, while the second possible implementation can obtain more accurate third light information, thereby achieving more accurate driving control.

[0090] In the above embodiment, the processor 120 obtains the first light information sent by the light detection device 110, which is only used as an example for description. However, this does not constitute any limitation to the present application. For example, the processor 120 may also obtain the first light information through a camera equipped with a filter component (such as the camera 130 equipped with the filter component 131 in Figure 1). The filter component 131 may be the same as the filter component 111 in the signal light detection device 110 in Figure 2. Alternatively, the processor 120 may obtain the first light information based on the color of the image captured by the camera.

[0091] Therefore, in the embodiment of the present application, accurate recognition of the information indicated by the signal light is achieved by filtering the signal light emitted by the signal light and converting the signal light into first light information indicating the recognition result of the signal light.

[0092] FIG4 is a flow chart of a vehicle control method provided by an embodiment of the present application. The embodiment shown in FIG4 may be executed by the processor 120 in FIG1 , and the method 200 includes some or all of the following processes:

[0093] S210, acquiring first light information from a signal light detection device, where the first light information is used to indicate a recognition result of a signal light emitted by a signal lamp in a target area;

[0094] S220, synchronously acquiring an image of the target area from the camera;

[0095] S230, identifying the image of the target area to obtain environmental information of the target area;

[0096] S240: Control the vehicle's driving according to the environmental information and the first light information.

[0097] In some embodiments, the driving control of the vehicle is performed based on the environmental information and the first light information, including: when the environmental information meets a preset condition, generating a driving instruction based on the first light information; and controlling the driving of the vehicle based on the driving instruction.

[0098] In some embodiments, the preset conditions include: the environmental information indicates that the light intensity of the target area is lower than the light threshold; and / or the environmental information indicates that the target area includes the signal light; and / or the recognition accuracy of the second light information indicated by the environmental information is less than the accuracy threshold, and the second light information is the recognition result of the signal light emitted by the signal light obtained by recognizing the image of the target area.

[0099] In some embodiments, vehicle driving control is performed based on environmental information and first light information, including: determining third light information through weighted operation based on the first light information and second light information in the environmental information, where the second light information is a recognition result of signal light emitted by a signal light obtained by recognizing an image of a target area; generating a driving instruction based on the third light information; and controlling vehicle driving based on the driving instruction.

[0100] In some embodiments, the driving instruction includes at least one of a driving instruction, a parking instruction, and a speed control instruction.

[0101] In some embodiments, an image of a target area is recognized to obtain environmental information of the target area, including: recognizing the image of the target area through a pre-trained visual model to obtain second light information, where the second light information is a recognition result of the signal light emitted by the signal light obtained by recognizing the image of the target area.

[0102] In some embodiments, the method 200 further includes: obtaining a plurality of training samples, where the training samples are images of signal lights transmitted through a filter, where the filter is used to transmit signal light emitted by the signal light; and training the initial network model based on the plurality of training samples to obtain a visual model.

[0103] In some embodiments, the method 200 further includes acquiring first light information through a camera, where the camera is equipped with a filter component.

[0104] FIG5 is a flow chart of another vehicle control method provided by an embodiment of the present application. The embodiment shown in FIG5 may be performed by the signal light detection device 120 in FIG1 or FIG2 . The method 300 may include some or all of the following processes:

[0105] S310, transmitting the signal light emitted by the signal lamp in the target area through the filter component;

[0106] S320, performing photoelectric conversion on the signal light to obtain first light information, where the first light information is used to indicate a recognition result of the signal light emitted by the signal lamp in the target area;

[0107] S330, output first light information.

[0108] In some embodiments, the filter assembly includes a plurality of filters, each of which is used to transmit signal light of different wavelengths emitted by a signal lamp in the target area, and the signal lamp emits signal light of one wavelength at the same time.

[0109] In the vehicle control method provided in the embodiment of the present application, the processor can obtain the recognition result of the signal light emitted by the signal light in the target area from the signal light detection device, that is, the first light information, and then combine the first light information with the environmental information based on image recognition to control the vehicle's driving, thereby improving the reliability of the driving control.

[0110] It should be understood that the various steps in the vehicle control method provided in FIG. 4 and FIG. 5 have been described in the embodiments shown in FIG. 1 to FIG. 3 , and for the sake of brevity, they will not be repeated here.

[0111] In the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between the various embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0112] The following describes in detail the device embodiment of the present application in conjunction with Figures 6 and 7. It should be understood that the device embodiment and the method embodiment correspond to each other, and similar descriptions can refer to the method embodiment.

[0113] FIG6 is a schematic block diagram of an apparatus 400 provided in an embodiment of the present application. As shown in FIG6 , the apparatus 400 can be implemented as the aforementioned vehicle control system, or as a chip or chip system in a vehicle control system, or as a processor in a chip (e.g., the processor 120 in FIG1 ). The apparatus 400 includes:

[0114] An acquiring unit 410 is configured to acquire first light information from a signal light detection device, where the first light information is used to indicate a recognition result of the signal light emitted by a signal lamp in a target area;

[0115] The acquisition unit 410 is also used to synchronously acquire an image of the target area from the camera;

[0116] The recognition unit 420 is used to recognize the image of the target area and obtain the environmental information of the target area;

[0117] The control unit 430 is configured to control the driving of the vehicle according to the environmental information and the first light information.

[0118] In some embodiments, the vehicle control unit 430 is specifically used to: generate a driving instruction according to the first light information when the environmental information meets the preset conditions; and perform driving control of the vehicle according to the driving instruction.

[0119] In some embodiments, the preset conditions include: the environmental information indicates that the light intensity of the target area is lower than the light threshold; and / or the environmental information indicates that the target area includes the signal light; and / or the recognition accuracy of the second light information indicated by the environmental information is less than the accuracy threshold, and the second light information is the recognition result of the signal light emitted by the signal light obtained by recognizing the image of the target area.

[0120] In some embodiments, the vehicle control unit 430 is specifically used to: determine the third light information through weighted operation based on the first light information and the second light information in the environmental information, where the second light information is the recognition result of the signal light emitted by the signal light obtained by recognizing the image of the target area; generate a driving instruction based on the third light information; and control the vehicle's driving according to the driving instruction.

[0121] In some embodiments, the driving instruction includes at least one of a driving instruction, a parking instruction, and a speed control instruction.

[0122] In some embodiments, the recognition unit 420 is specifically used to: recognize the image of the target area through a pre-trained visual model to obtain second light information, where the second light information is the recognition result of the signal light emitted by the signal light obtained by recognizing the image of the target area.

[0123] In some embodiments, the acquisition unit 410 is further used to: acquire multiple training samples, where the training samples are images of signal lights transmitted through a filter, where the filter is used to transmit signal light emitted by the signal light; and train the initial network model based on the multiple training samples to obtain a visual model.

[0124] In some embodiments, the acquisition unit 410 is further configured to: acquire the first light information through a camera, where the camera is equipped with a filter component.

[0125] It should be understood that the various units in the device 400 according to the embodiment of the present application are respectively for implementing the corresponding processes of the processor in the method shown in Figure 3. For the sake of brevity, they are not repeated here.

[0126] Figure 7 is a schematic structural diagram of an apparatus 500 according to an embodiment of the present application. The apparatus 500 shown in Figure 7 includes a processor 510, which can call and execute a computer program from a memory to implement the method according to the embodiment of the present application.

[0127] In some embodiments, as shown in FIG7 , the apparatus 500 may further include a memory 520 , wherein the processor 510 may call and execute a computer program from the memory 500 to implement the method in the embodiment of the present application.

[0128] The memory 520 may be a separate device independent of the processor 510 , or may be integrated into the processor 510 .

[0129] In some embodiments, the apparatus 500 may further include an input interface 530. The processor 510 may control the input interface 530 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0130] In some embodiments, the apparatus 500 may further include an output interface 540. The processor 510 may control the output interface 540 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0131] In some embodiments, the device 500 can implement the corresponding processes of the various methods of the embodiments of the present application. For the sake of brevity, they are not described here.

[0132] In some embodiments, the device mentioned in the embodiments of the present application may also be a chip, such as a system-on-chip, a system-on-chip, a chip system, or a system-on-chip chip.

[0133] It should be understood that the processor of the embodiments of the present application may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiment can be completed by hardware integrated logic circuits in the processor or software instructions. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0134] It is understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0135] It should be understood that the above-mentioned memories are exemplary but not restrictive. For example, the memories in the embodiments of the present application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM RAM (DR RAM), etc. In other words, the memories in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0136] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0137] In some embodiments, the computer-readable storage medium can be applied to the processor in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes in the various methods in the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0138] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0139] In some embodiments, the computer program product can be applied to the processor in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes in the various methods in the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0140] The embodiment of the present application also provides a computer program.

[0141] In some embodiments, the computer program can be applied to the processor in the embodiments of the present application. When the computer program runs on a computer, the computer executes the corresponding processes in the various methods in the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0142] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0143] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0144] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0145] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0146] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. In view of this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0147] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A signal light detection device, characterized in that: include: Optical filter components and photodetectors; The filter assembly is used to transmit the signal light emitted by the signal lamp; The photodetector is used to convert the signal light into first light information and output the first light information, where the first light information is used to indicate a recognition result of the signal light.

2. The device according to claim 1, characterized in that The filter assembly includes a plurality of filters, each of which is used to transmit signal light of different wavelengths. The signal light emits signal light of one wavelength at the same time.

3. The device according to claim 1 or 2, characterized in that The device further comprises: an optical fiber jumper; the optical fiber jumper is used to establish an optical transmission channel between each filter in the filter assembly and the photodetector; The optical fiber jumper transmits the signal light output by the filter assembly to the photoelectric detector through the corresponding optical transmission channel.

4. The device according to any one of claims 1 to 3, characterized in that The filter assembly is disposed on the inner side of the windshield of the vehicle.

5. A vehicle control method, characterized in that: include: acquiring first light information from a signal light detection device, where the first light information is used to indicate a recognition result of the signal light emitted by a signal lamp in a target area; Synchronously acquire images of the target area from the camera; Recognizing the image of the target area to obtain environmental information of the target area; The vehicle driving is controlled according to the environmental information and the first light information.

6. The method according to claim 5, characterized in that The performing driving control of the vehicle according to the environmental information and the first light information includes: When the environmental information meets a preset condition, generating a driving instruction according to the first light information; The vehicle driving is controlled according to the driving instruction.

7. The method according to claim 6, characterized in that The preset conditions include: The environmental information indicates that the light intensity of the target area is lower than a light threshold; and / or, The environmental information indicates that the target area includes the traffic light; and / or, The recognition accuracy of the second light information indicated by the environmental information is less than an accuracy threshold, and the second light information is a recognition result of the signal light emitted by the signal light obtained by recognizing the image of the target area.

8. The method according to claim 5, characterized in that The performing driving control of the vehicle according to the environmental information and the first light information includes: determining third light information by weighted calculation based on the first light information and second light information in the environmental information, wherein the second light information is a recognition result of the signal light emitted by the signal light obtained by recognizing the image of the target area; generating a driving instruction according to the third light information; The vehicle driving is controlled according to the driving instruction.

9. The method according to any one of claims 6 to 8, characterized in that The driving instruction includes at least one of a driving instruction, a parking instruction, and a speed control instruction.

10. The method according to any one of claims 5 to 9, characterized in that The step of identifying the image of the target area to obtain environmental information of the target area includes: The image of the target area is recognized by a pre-trained visual model to obtain second light information, where the second light information is a recognition result of the signal light emitted by the signal light obtained by recognizing the image of the target area.

11. The method according to claim 10, characterized in that The method further comprises: Acquire a plurality of training samples, where the training samples are images of signal lights transmitted through a filter, where the filter is used to transmit signal light emitted by the signal lights; The initial network model is trained according to the multiple training samples to obtain the visual model.

12. A vehicle control method, characterized in that: include: transmitting the signal light emitted by the signal lamp in the target area through the filter component; Performing photoelectric conversion on the signal light to obtain first light information, where the first light information is used to indicate a recognition result of the signal light emitted by the signal lamp in the target area; The first light information is output.

13. The method according to claim 12, characterized in that The filter assembly includes a plurality of filters, each of which is used to transmit signal lights of different wavelengths emitted by a signal lamp in the target area. The signal lamp emits signal light of one wavelength at the same time.

14. A chip, characterized in that: The device comprises a processor configured to call and execute a computer program from a memory, so that a device equipped with the chip executes the method according to any one of claims 5 to 13.

15. A vehicle control system, characterized in that: include: The signal light detection device according to any one of claims 1 to 4 and the chip according to claim 14.