Traffic signal light image processing method and device, and edge computing device

Abstract

The disclosure provides a traffic signal lamp image processing method and device and an edge computing device, relates to the technical field of intelligent transportation, and in particular to the field of image processing. The implementation scheme is: acquiring a first image of a signal lamp and performing thresholding processing on the first image to obtain at least one first centroid; acquiring a first image set, the first image set including at least a second image taken after the first image; performing thresholding processing on the second image to obtain at least one second centroid; for each of the at least one first centroid, in response to determining that the first centroid corresponds to one of the at least one second centroid, determining the first centroid as a candidate centroid; and acquiring a lamp head centroid corresponding to a lamp head of the signal lamp from one or more candidate centroids in the at least one first centroid.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent transportation technology, in particular to image processing technology, and specifically to a method, apparatus, edge computing device, electronic device, computer-readable storage medium, and computer program product for processing traffic light images. Background Art

[0002] Roadside sensing involves deploying sensors on the side of the road to analyze information on the road. For example, in roadside traffic light sensing technology, cameras are installed at fixed locations on the road to capture image data, which is then analyzed to obtain information about the road.

[0003] In the roadside traffic light perception solution, the position of the traffic light relative to the camera remains unchanged, ensuring that the position of the traffic light in the image captured by the camera is fixed. After the traffic light position is annotated based on the initial image captured by the camera, the position of the traffic light in subsequent images captured by the camera can be determined based on the annotated initial image.

[0004] However, when the traffic light moves slightly (such as thermal expansion and contraction, ground sinking), or the camera device moves slightly (the camera device is loosely installed), the position between the camera device and the traffic light will change. At this time, the position of the traffic light in the initial image may be different from the position of the traffic light in the currently captured image, making it impossible to obtain the exact position of the traffic light in the currently captured image based on the annotated initial image.

[0005] The approaches described in this section are not necessarily approaches that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any approach described in this section is prior art simply by virtue of its inclusion in this section. Similarly, unless otherwise indicated, the issues raised in this section should not be considered as having been recognized in any prior art. Summary of the Invention

[0006] The present disclosure provides a method, apparatus, edge computing device, electronic device, computer-readable storage medium, and computer program product for processing traffic light images.

[0007] According to one aspect of the present disclosure, a method for processing a traffic light image is provided, comprising: acquiring a first image of the traffic light and performing thresholding processing on the first image to obtain at least one first centroid; acquiring a first image set, the first image set including at least a second image taken after the first image; performing the thresholding processing on the second image to obtain at least one second centroid; for each first centroid of the at least one first centroid, in response to determining that the first centroid corresponds to one of the at least one second centroid, determining the first centroid as a candidate centroid; and acquiring a lamp head centroid corresponding to the lamp head of the traffic light from one or more candidate centroids of the at least one first centroid.

[0008] According to another aspect of the present disclosure, a device for processing traffic light images is provided, comprising: a first acquisition unit configured to acquire a first image of the traffic light and perform thresholding processing on the first image to obtain at least one first centroid; a second acquisition unit configured to acquire a first image set, the first image set including at least a second image taken after the first image; a thresholding processing unit configured to perform the thresholding processing on the second image to obtain at least one second centroid; a determination unit configured to determine, for each first centroid of the at least one first centroid, the first centroid as a candidate centroid in response to determining that the first centroid corresponds to one of the at least one second centroid; and a third acquisition unit configured to acquire a lamp head centroid corresponding to the lamp head of the traffic light from one or more candidate centroids among the at least one first centroid.

[0009] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to implement the above-mentioned method.

[0010] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to implement the above method.

[0011] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0012] According to another aspect of the present disclosure, the electronic device as described above is included.

[0013] According to one or more embodiments of the present disclosure, a single-frame image is subjected to thresholding processing to obtain at least one centroid within the single-frame image. The position of the signal light head corresponding to the centroid is determined by determining the centroid of the signal light head within the at least one centroid obtained. Because the thresholding processing is performed on the entire single-frame image, there is no need to define ROI boundaries and, therefore, is unaffected by changes in vehicle lights and lighting conditions. Furthermore, the position of the signal light head can be determined even in the presence of distant vehicle lights and when the signal light color remains unchanged.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings illustrate exemplary embodiments and constitute a part of the specification. Together with the description of the specification, they serve to explain exemplary implementation of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals designate similar, but not necessarily identical, elements.

[0016] Figure 1 A schematic diagram illustrating an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;

[0017] Figure 2 A flowchart of a method for processing a signal light image according to an embodiment of the present disclosure is shown;

[0018] Figure 3 A flowchart showing a process of performing a first thresholding process on the first image in the method for processing a signal light image according to an embodiment of the present disclosure is shown;

[0019] Figure 4 A flowchart of a process for corresponding each first centroid of at least one centroid to one of at least one second centroid in a method for processing a traffic light image according to an embodiment of the present disclosure is shown;

[0020] Figure 5 A flowchart of a method for processing a signal light image according to an embodiment of the present disclosure is shown;

[0021] Figure 6 A flowchart of a method for processing a signal light image according to an embodiment of the present disclosure is shown;

[0022] Figure 7A flowchart showing a process of determining the centroid of a lamp head based on a third image in a method for processing a signal light image according to an embodiment of the present disclosure is shown;

[0023] Figure 8 A structural block diagram of a device for processing a signal light image according to an embodiment of the present disclosure; and

[0024] Figure 9 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0025] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0026] In this disclosure, unless otherwise specified, the use of terms such as "first" and "second" to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements. Such terms are only used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of the element, while in some cases, based on the context of the description, they may also refer to different instances.

[0027] The terms used in the descriptions of the various examples described in this disclosure are for the purpose of describing specific examples only and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element may be one or more. In addition, the term "and / or" used in this disclosure encompasses any one and all possible combinations of the listed items.

[0028] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0029] Figure 1 FIG2 is a schematic diagram of an exemplary system 100 in which the various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes a motor vehicle 110 , a server 120 , and one or more communication networks 130 coupling the motor vehicle 110 to the server 120 .

[0030] In an embodiment of the present disclosure, the motor vehicle 110 may include a computing device according to an embodiment of the present disclosure and / or be configured to perform a method according to an embodiment of the present disclosure.

[0031] The server 120 may run one or more services or software applications that enable the execution of the signal light image processing method disclosed herein. In some embodiments, the server 120 may also provide other services or software applications that may include non-virtual environments and virtual environments. Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. The user of the motor vehicle 110 may, in turn, utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0032] Server 120 may include one or more general-purpose computers, specialized server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0033] The computing units in the server 120 may run one or more operating systems including any of the operating systems described above as well as any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.

[0034] In some embodiments, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from motor vehicle 110. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of motor vehicle 110.

[0035] The network 130 may be any type of network known to those skilled in the art that can support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a satellite communication network, a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (including, for example, Bluetooth, WiFi), and / or any combination of these and other networks.

[0036] The system 100 may also include one or more databases 150. In some embodiments, these databases can be used to store data and other information. For example, one or more of the databases 150 can be used to store information such as audio files and video files. The data repository 150 can reside in a variety of locations. For example, the data repository used by the server 120 can be local to the server 120, or can be remote from the server 120 and can communicate with the server 120 via a network-based or dedicated connection. The data repository 150 can be of different types. In some embodiments, the data repository used by the server 120 can be a database, such as a relational database. One or more of these databases can store, update, and retrieve data to and from the database in response to commands.

[0037] In some embodiments, one or more of the databases 150 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0038] Motor vehicle 110 may include sensors 111 for sensing its surroundings. Sensors 111 may include one or more of the following: visual cameras, infrared cameras, ultrasonic sensors, millimeter-wave radar, and laser radar (LiDAR). Different sensors offer different detection accuracy and range. Cameras may be mounted on the front, rear, or other locations of the vehicle. Visual cameras can capture real-time information about the vehicle's interior and exterior and present it to the driver and / or passengers. Furthermore, by analyzing the images captured by the visual cameras, information such as traffic light indications, intersection conditions, and the operating status of other vehicles can be obtained. Infrared cameras can detect objects in night vision conditions. Ultrasonic sensors can be mounted on all sides of the vehicle, utilizing the strong directionality of ultrasonic waves to measure the distance of external objects from the vehicle. Millimeter-wave radars can be mounted on the front, rear, or other locations of the vehicle, utilizing the properties of electromagnetic waves to measure the distance of external objects from the vehicle. LiDARs can be mounted on the front, rear, or other locations of the vehicle, detecting object edges and shapes for object recognition and tracking. Due to the Doppler effect, radar devices can also measure changes in the speed of the vehicle and moving objects.

[0039] The motor vehicle 110 may also include a communication device 112. The communication device 112 may include a satellite positioning module that can receive satellite positioning signals (e.g., Beidou, GPS, GLONASS, and GALILEO) from satellites 141 and generate coordinates based on these signals. The communication device 112 may also include a module for communicating with a mobile communication base station 142. The mobile communication network may implement any suitable communication technology, such as GSM / GPRS, CDMA, LTE, and other current or evolving wireless communication technologies (e.g., 5G technology). The communication device 112 may also have a vehicle-to-everything (V2X) module that is configured to implement vehicle-to-vehicle (V2V) communication with other vehicles 143 and vehicle-to-infrastructure (V2I) communication with infrastructure 144, for example. In addition, the communication device 112 may also include a module configured to communicate with a user terminal 145 (including but not limited to a smartphone, tablet computer, or wearable device such as a watch) via a wireless local area network or Bluetooth using the IEEE 802.11 standard, for example. Using the communication device 112, the motor vehicle 110 may also access the server 120 via the network 130.

[0040] The motor vehicle 110 may also include a control device 113. The control device 113 may include a processor that communicates with various types of computer-readable storage devices or media, such as a central processing unit (CPU) or a graphics processing unit (GPU), or other dedicated processors. The control device 113 may include an autonomous driving system for automatically controlling various actuators in the vehicle. The autonomous driving system is configured to control the powertrain, steering system, and braking system of the motor vehicle 110 (not shown) via multiple actuators in response to input from multiple sensors 111 or other input devices to control acceleration, steering, and braking, respectively, without human intervention or limited human intervention. Some processing functions of the control device 113 may be implemented through cloud computing. For example, some processing may be performed using an on-board processor, while other processing may be performed using computing resources in the cloud. The control device 113 may be configured to execute the method according to the present disclosure. In addition, the control device 113 may be implemented as an example of a computing device on the motor vehicle side (client) according to the present disclosure.

[0041] Figure 1 The system 100 may be configured and operated in various ways to enable application of the various methods and apparatuses described in accordance with the present disclosure.

[0042] See Figure 2 According to some embodiments of the present disclosure, a signal light image processing 200 includes:

[0043] Step S210: Acquire a first image of the traffic light and perform thresholding processing on the first image to obtain at least one first centroid;

[0044] Step S220: Acquire a first image set, where the first image set at least includes a second image captured after the first image.

[0045] Step S230: performing the thresholding process on the second image to obtain at least one second centroid;

[0046] Step S240: For each first centroid of the at least one first centroid,

[0047] In response to determining that the first centroid corresponds to one of the at least one second centroid, determining the first centroid as a candidate centroid; and

[0048] Step S250: Acquire a lamp head mass center corresponding to the lamp head of the signal lamp from one or more candidate mass centers among the at least one first mass center.

[0049] In related art, methods for obtaining the position of a signal light's head based on a ROI (Route of Interest) from a historical template involve obtaining an ROI region in an image based on the ground. The ROI region is then thresholded to obtain its centroid. The centroid of the ROI region in the current image is then compared with the centroid of the ROI region in the historical template to determine the offset of the signal light's head. However, at night, the ROI region obtained relative to the ground is often affected by vehicle lights and changing lighting conditions, making the obtained ROI region inaccurate. This makes it difficult to calculate the overall offset of the signal light's head and therefore unusable. Furthermore, when vehicles are in the distance, the colors of the vehicle lights and the signal light become confused, often making it unclear whether the obtained centroid of the ROI region corresponds to the centroid of the signal light's head.

[0050] According to one or more embodiments of the present disclosure, a single-frame image is subjected to thresholding processing to obtain at least one centroid within the single-frame image. The position of the signal light head corresponding to the centroid is determined by determining the centroid of the signal light head within the at least one centroid obtained. Because the thresholding processing is performed on the entire single-frame image, there is no need to define ROI boundaries and, therefore, is unaffected by changes in vehicle lights and lighting conditions. Furthermore, the position of the signal light head can be determined even in the presence of distant vehicle lights and when the signal light color remains unchanged.

[0051] At the same time, according to the method disclosed in the present invention, the method of obtaining the image centroid through image thresholding processing has less data processing and is simpler than the method of obtaining the characteristic points of the traffic light in the image to obtain the position of the traffic light head by extracting image features.

[0052] In some embodiments, the signal light is a traffic light installed on the road.

[0053] In some embodiments, the signal light includes a light head that has different colors at different times.

[0054] In some embodiments, the signal light includes at least two lamp heads, such as a red light lamp head and a green light lamp head. In other embodiments, the signal light also includes an amber light lamp head.

[0055] In some embodiments, the first image of the traffic light is acquired by a camera installed at a fixed position on the road.

[0056] In some embodiments, the first image is obtained from a time-varying image sequence acquired by a camera device. For example, the camera device periodically acquires images to obtain the image sequence. In another example, the camera device continuously acquires road video to obtain the image sequence.

[0057] In some embodiments, the first image is acquired in real time by a camera that acquires a sequence of images that changes over time.

[0058] In some embodiments, as Figure 3 As shown, performing thresholding processing on the first image to obtain at least one centroid of the image includes:

[0059] Step S310: performing binarization processing on the first image based on a first preset threshold value to obtain a binary image, wherein the first preset threshold value corresponds to the color of the lamp head when it is lit; and

[0060] Step S320: Based on the binarized image, obtain the at least one first centroid.

[0061] When performing the first thresholding process on the first image, the first preset threshold is made to correspond to the color of the lamp head of the traffic light, so that the area corresponding to the color of the lamp head in the first image is highlighted, the interference of the background area is reduced, and the obtained centroid is accurate.

[0062] In some embodiments, the first threshold is a numerical value. For example, if the first threshold is slightly smaller than the grayscale value corresponding to the color of the red light head when it is illuminated, the grayscale value of the area where the corresponding grayscale value is higher than the first threshold is set to 255, and the grayscale value of the area where the corresponding grayscale value is higher than the first threshold is set to 0 (or, the grayscale value of the area where the corresponding grayscale value is higher than the first threshold is set to 0, and the grayscale value of the area where the corresponding grayscale value is higher than the first threshold is set to 255).

[0063] In some embodiments, the first preset threshold corresponds to a threshold range defined by two vertical boundaries. For example, the first preset threshold corresponds to a grayscale value range that includes the grayscale value corresponding to the color when the red light head is illuminated (e.g., a range between grayscale values ​​100 and grayscale values ​​130), and the grayscale value of a first region of the first image within the grayscale value range having the grayscale value of the corresponding color is set to 255, and the grayscale value of a second region outside the first region of the first image is set to 0 (or conversely, the grayscale value of the first region is set to 0 and the grayscale value of the second region is set to 255).

[0064] It should be understood that in the above embodiment, the first threshold value corresponds to the color when the red light head is lit. This is merely exemplary. In other embodiments, the first threshold value may also correspond to the color when the green light head is lit. This is not limited here.

[0065] In some embodiments, after obtaining a binarized image based on the first threshold, an opening operation is performed on the obtained binarized image to fill holes and remove some independent noise points therein.

[0066] In some embodiments, thresholding the first image further includes performing connected domain analysis on the binarized image to obtain the contour of each connected domain in at least one connected domain in the binarized image; and obtaining the at least one centroid based on the contour of each connected domain in the at least one connected domain.

[0067] After binarization of the image, the contour of the connected domain in the image is obtained, and at least one centroid is obtained based on the contour. The threshold processing process is simple and the amount of data processing is small.

[0068] In some embodiments, the first image set includes a single image, namely, a second image captured after the first image. In some embodiments, the second image is an image that follows the first image in a time-varying image sequence. In some embodiments, the second image and the first image are two frames captured when the same lamp head is illuminated.

[0069] The method for performing thresholding processing on the second image is the same as the method for performing thresholding processing on the first image. It should be noted that, unless otherwise specified, the thresholding processing of the present disclosure adopts the same method.

[0070] In step S240 , for each first centroid among the at least one first centroid, in response to determining that the first centroid corresponds to one of the at least one second centroid, the first centroid is determined as a candidate centroid.

[0071] Because the second image was captured after the first image, if one of the second centroids is determined to correspond to a first centroid in the first image, this indicates that the object (headlight or signal light) corresponding to this first centroid has remained essentially unchanged between the capture of the first and second images. This means the corresponding object could be a headlight that was illuminated in both the first and second images, or a headlight of a currently stationary vehicle. Therefore, this first centroid is identified as a candidate centroid to further determine whether it corresponds to a signal light.

[0072] In some cases, the vehicle is in a moving state. By acquiring a second image after the first image and acquiring one or more candidate centroids from at least one first centroid in the first image, the first centroid corresponding to the headlights of the moving vehicle in the first image can be excluded, so that the range of the obtained one or more candidate centroids is as small as possible, thereby reducing the amount of subsequent data processing.

[0073] In some embodiments, for each first centroid among the at least one first centroid, the first centroid is determined to correspond to one of the at least one second centroid by determining that a pixel area of ​​the first centroid in the first image coincides with a pixel area of ​​one of the at least one second centroid in the second image.

[0074] In some embodiments, as Figure 4 As shown, for each first center of mass of the at least one first center of mass, determining that the first center of mass corresponds to one of the at least one second center of mass includes:

[0075] Step S410: Obtain at least one first position corresponding one-to-one to the at least one first mass center and at least one second position corresponding one-to-one to the at least one second mass center;

[0076] Step S420: For each first centroid of the at least one first centroid, obtaining at least one distance from the first centroid to the at least one second centroid; and

[0077] Step S430: For each first centroid among the at least one first centroid, in response to determining that the distance from the first centroid to a second centroid among the at least one second centroid is less than a preset distance threshold, determine that the first centroid corresponds to the second centroid.

[0078] In some cases, the camera device may experience slight vibration or shaking due to a loose fixture, causing the position of the same lamp head to change between two images captured at adjacent time points. By comparing the positions of at least one first center of mass and at least one second center of mass, and determining that the first and second centers of mass whose distance is less than a preset distance threshold are corresponding, the center of mass corresponding to the lamp head can be effectively found.

[0079] In some embodiments, the first position and the second position are the coordinate positions of the two corresponding centroids in the same spatial coordinate system. By calculating the distance between the two coordinate positions, the distance between the two corresponding centroids in space is calculated.

[0080] In some embodiments, for each first center of mass among the at least one first center of mass, in response to determining that the first center of mass does not correspond to any second center of mass among the at least one second center of mass, the image is reacquired until a second image having a second center of mass corresponding to one of the first centers of mass is acquired.

[0081] For example, when the traffic light changes from a green light head to a red light head, a first image and an adjacent image adjacent to it are acquired. In the first image, the green light head is lit, and in the adjacent image, the red light head is lit, so that the center of mass in the first image cannot find a corresponding center of mass in the adjacent image. Then, the image after the adjacent image is acquired until an image with a center of mass corresponding to the center of mass in the first image is acquired.

[0082] In some embodiments, for each first centroid of the at least one first centroid, in response to determining that the first centroid does not correspond to any second centroid of the at least one second centroid, the first image and the second image are reacquired until two adjacent images with corresponding centroids are acquired, and the two adjacent images are respectively used as the first image and the second image.

[0083] In some embodiments, each image in the first image set is taken within a preset time range after the first image, and wherein the preset time range corresponds to the duration of time when the lamp head is illuminated.

[0084] By acquiring the first image and the first image set within the time when the lamp head is lit, since the time interval between the images in the first image set and the first image is small (less than the time when the lamp head is lit), it is possible to determine that the one or more candidate centroids include the lamp head centroid corresponding to the lamp head, thereby making it possible to accurately obtain one or more candidate centroids based on the first image and the first image set.

[0085] In some embodiments, the first image set includes the plurality of images, ie, includes the plurality of images in addition to a second image taken after the first image.

[0086] In some embodiments, by performing thresholding processing on each of the multiple images, at least one third center of mass corresponding to each of the multiple images is obtained, and for each first center of mass in the at least one first center of mass, in response to determining that at least one third center of mass in each of the multiple images includes a third center of mass corresponding to the first center of mass, the first center of mass is determined as a candidate center of mass.

[0087] For example, during the duration of a red light being on (e.g., 10 seconds), a first image set is acquired every 1 second after the first image, and the first image set includes eight images excluding the second image. For each first centroid of the at least one first centroid in the first image, if it is determined that the first centroid has a corresponding centroid in the second image and each of the eight images, the first centroid is determined to be a candidate centroid.

[0088] By determining one or more candidate centroids from the at least one centroid, in a further processing step, a lamp head centroid from the one or more candidate centroids corresponding to the lamp head of the signal lamp is determined.

[0089] In one embodiment, the lamp head centroid is determined by determining the number of one or more candidate centroids.

[0090] See Figure 5 A flowchart of a method 500 for processing a signal light image according to some embodiments of the present disclosure is shown, wherein steps S510 to S550 refer to the above description of the signal light image processing method 500. Figure 2 The description of steps S210-S250 of the signal light image processing method 200 is omitted here. Figure 5 As shown, in some embodiments, the method 500 further includes:

[0091] Step S560: Determine the number of the one or more candidate centroids;

[0092] Step S570: In response to determining that the number of the one or more candidate mass centers is equal to 1, determining the one or more candidate mass centers as the lamp head mass center; and

[0093] Step S570: In response to determining that the number of the one or more candidate centroids is not equal to 1, acquiring a third image, and acquiring the lamp head centroid based on the third image.

[0094] Through thresholding, when only one centroid is identified as a candidate centroid in the image, since the image is taken of a traffic light, the candidate centroid is the lamp head centroid. This method simplifies the method for determining the center of mass of the traffic light head and reduces the amount of data processing.

[0095] When the number of one or more candidate centroids is not equal to 1 (i.e., greater than 1), it is necessary to determine the centroid corresponding to the lamp head centroid from the multiple candidate centroids. In this case, a third image is further acquired to obtain the lamp head centroid corresponding to the lamp head from the multiple candidate centroids.

[0096] In some embodiments, the third image may be an image captured at a first interval from the time the first image was captured, where the first interval is the duration of time the corresponding lamp head in the first image is illuminated. A thresholding process is performed on the third image to obtain at least one third centroid of the third image, and in response to determining that one of the one or more candidate centroids corresponds to one of the at least one third centroids, the candidate centroid is determined to be the centroid of the lamp head.

[0097] For example, in the first image, the green light is on, and the green light is on for 15 seconds. Then, the third image is an image obtained within the 15 seconds when the green light is on.

[0098] In some embodiments, the shooting time of the third image is separated from the shooting time of the first image by a preset time interval, and the preset time interval is at least greater than the duration of the lighting of the lamp head, and Figure 6 As shown, obtaining the centroid of the lamp head based on the third image includes:

[0099] Step S610: performing difference processing on the first image and the third image to obtain a difference image;

[0100] Step S620: performing the thresholding process on the difference image to obtain a difference centroid of the difference image; and

[0101] Step S630: In response to determining that the differential centroid corresponds to one of the one or more candidate centroids, determine the one candidate centroid as the lamp head centroid.

[0102] The interval between the capture time of the third image and the capture time of the first image is at least greater than the duration of the lamp head being illuminated. That is, the third image is captured outside the duration of the lamp head being illuminated. That is, the illuminated lamp color in the third image is different from the illuminated lamp color in the first image. For example, if the red lamp head is illuminated in the first image and the red lamp head is illuminated for 10 seconds, then the third image is captured 10 seconds after the first image. That is, in the third image, the yellow or green lamp head is illuminated.

[0103] A differential image is obtained by performing a difference process on the first and third images, wherein the pixel value of each pixel in the differential image corresponds to the difference value of the corresponding pixel in the first and third images. A second thresholding process is then performed on the differential image to obtain a differential centroid, which corresponds to the centroid of the lit lamp head in the first image. Based on the differential centroid, a lamp head centroid from one or more candidate centroids can be obtained.

[0104] By acquiring a third image whose time interval between the above-mentioned shooting time and the shooting time of the first image is a preset time interval, and the preset time interval is at least greater than the duration of the lamp head being lit, the background part in the differential image obtained based on the third image and the first image is further reduced, and the obtained differential centroid is accurate, thereby making the lamp head centroid among one or more candidate centroids determined based on the differential centroid more accurate.

[0105] In the related art, a differential image is obtained by performing differential processing on two frames of images in which the light color of a traffic light has changed, and the differential image is thresholded to obtain a centroid of the differential image, which is the centroid of the lamp head of the traffic light after the light color has changed. However, in this technology, the environment in the two frames of images that need to be obtained does not change and the light color does not change, and the requirements for the obtained images are stringent. For example, when there is a vehicle in the distance, the change in the color of the vehicle light can easily cause changes in adjacent frames, making it difficult to distinguish the centroid of the vehicle light and the lamp head of the traffic light calculated based on the difference between the frames, resulting in an error in the determined centroid of the lamp head; and, when the lamp head of the traffic light does not change, the centroid of the traffic light cannot be obtained. According to the embodiments of the present disclosure, the centroid of the lamp head can be accurately obtained under any circumstances, the requirements for the image obtained are low, and it can be widely applied to various environments and situations.

[0106] In some embodiments, after determining one or more first candidate centroids, a third image is directly acquired to obtain the centroid of the lamp head from the one or more candidate centroids. Figure 7 , shows a flowchart of a method 700 for processing a traffic light image according to some embodiments of the present disclosure, wherein steps S710-S750 in method 700 refer to the above description of steps S210-S250 in method 200, and step S760 refers to the above description of steps S610-S630, which are not repeated here.

[0107] In some embodiments, after the centroid of the lamp head among the one or more candidate centroids is determined, the position of the lamp head is determined using the determined centroid of the lamp head.

[0108] For example, determining the lamp head position may include: obtaining a position corresponding to the lamp head centroid in the first image based on the lamp head centroid and a pre-calibrated lamp head size. The lamp head size may be, for example, a lamp head in a signal lamp calibrated based on a pre-obtained signal lamp image.

[0109] In some embodiments, each image in the first image set includes a centroid corresponding to the centroid of the lamp head, and the position of the lamp head is determined based on the centroid of the lamp head and the corresponding centroid corresponding to the centroid of the lamp head in each image in the first image set.

[0110] For example, the position of the lamp head is determined based on the average position of the centroid of the lamp head and the corresponding centroids in each image in the first image set.

[0111] The above processing process makes the determined lamp head position more accurate.

[0112] According to another aspect of the present disclosure, an object recommendation device is also provided. Figure 8The device 800 includes: a first acquisition unit 810, configured to acquire a first image of the traffic light and perform a first thresholding process on the first image to obtain at least one first centroid; and a second acquisition unit 820, configured to acquire a first image set, the first image set including at least a second image taken after the first image; a first thresholding process unit 830, configured to perform the first thresholding process on the second image to obtain at least one second centroid; a determination unit 840, configured to determine, for each first centroid in the at least one first centroid, the first centroid as a candidate centroid in response to determining that the first centroid corresponds to one of the at least one second centroid; and a lamp head centroid acquisition unit 850, configured to acquire a lamp head centroid corresponding to the lamp head of the traffic light from one or more candidate centroids in the at least one first centroid.

[0113] In some embodiments, the first acquisition unit further includes: a binarization unit, configured to perform binarization processing on the first image based on a first preset threshold to obtain a binarized image, wherein the first preset threshold corresponds to the color of the lamp head when it is lit; and a first acquisition sub-unit, configured to obtain the at least one first centroid based on the binarized image.

[0114] In some embodiments, each image in the first image set is taken within a preset time range after the first image, and wherein the preset time range corresponds to the duration of time when the lamp head is illuminated.

[0115] In some embodiments, the determination unit 840 includes: a third acquisition unit, configured to obtain at least one first position corresponding one-to-one to the at least one first center of mass and at least one second position corresponding one-to-one to the at least one second center of mass; a fourth acquisition unit, configured to obtain, for each first center of mass in the at least one first center of mass, at least one distance from the first center of mass to the at least one second center of mass; and a determination subunit, configured to determine that, for each first center of mass in the at least one first center of mass, the first center of mass corresponds to the second center of mass in response to determining that the distance from the first center of mass to a second center of mass in the at least one second center of mass is less than a preset distance threshold.

[0116] In some embodiments, the lamp head center of mass acquisition 850 unit includes: a quantity determination unit, configured to determine the quantity of the one or more candidate centers of mass; a lamp head center of mass determination subunit, configured to determine the one or more candidate centers of mass as the lamp head center of mass in response to determining that the quantity of the one or more candidate centers of mass is equal to 1; and a third image acquisition unit, configured to acquire a third image in response to determining that the quantity of the one or more candidate centers of mass is greater than 1, and acquire the lamp head center of mass based on the third image.

[0117] In some embodiments, the shooting time of the third image is separated from the shooting time of the first image by a preset time interval, and the preset time interval is at least greater than the duration of the lamp head being lit, and the third image acquisition unit includes: a differential processing unit, configured to perform differential processing on the first image and the third image to obtain a differential image; a second threshold processing unit, configured to perform second threshold processing on the differential image to obtain at least one candidate differential centroid of the differential image; and a matching unit, configured to determine that one of the at least one differential centroid corresponds to one of the one or more candidate centroids, and to determine that the candidate centroid is the lamp head centroid.

[0118] In some embodiments, the apparatus 800 further includes: a lamp head position acquisition unit configured to acquire the position of the lamp head based on the lamp head center of mass and a corresponding center of mass among the at least one second center of mass corresponding to the lamp head center of mass.

[0119] According to another aspect of the present disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program, and the computer program implements the above method when executed by the at least one processor.

[0120] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing a computer program is further provided, wherein the computer program implements the above method when executed by a processor.

[0121] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0122] According to another aspect of the present disclosure, an edge computing device is also provided. Optionally, the edge computing device may include not only electronic devices but also communication components, and the electronic devices and communication components may be integrated or provided separately. The electronic devices can obtain data, such as images and videos, from roadside sensing devices (such as roadside cameras), perform image and video processing and data calculations, and then transmit the processing and calculation results to the cloud control platform via the communication components.

[0123] Optionally, the edge computing device can also be a roadside computing unit (RSCU). Optionally, the electronic device itself can also have sensory data acquisition and communication functions, such as an AI camera. The electronic device can directly perform image and video processing and data calculation based on the acquired sensory data, and then transmit the processing and calculation results to the cloud control platform.

[0124] Optionally, the cloud control platform performs processing in the cloud, including image and video processing and data calculation. The cloud control platform may also be referred to as a vehicle-road collaborative management platform, a V2X platform, a cloud computing platform, a central system, a cloud server, etc. According to an embodiment of the present disclosure, an electronic device, a readable storage medium, and a computer program product are also provided.

[0125] refer to Figure 9 , a block diagram of an electronic device 900 that can serve as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0126] like Figure 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the device 900 can also be stored in the RAM 903. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0127] Multiple components within device 900 are connected to I / O interface 905, including an input unit 906, an output unit 907, a storage unit 908, and a communication unit 909. Input unit 906 can be any type of device capable of inputting information into device 900. Input unit 906 can receive input numeric or character information and generate key signal input related to user settings and / or function control of the electronic device. It may include, but is not limited to, a mouse, keyboard, touch screen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 907 can be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, an audio / visual output terminal, a vibrator, and / or a printer. Storage unit 908 may include, but is not limited to, a magnetic disk or an optical disk. Communication unit 909 allows device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks. It may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and / or chipset, such as a Bluetooth™ device, a 1302.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0128] The computing unit 901 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as method 200. For example, in some embodiments, method 200 can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the method 200 described above can be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform the method 200 in any other appropriate manner (e.g., by means of firmware).

[0129] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0130] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0131] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0133] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0134] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0135] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0136] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but is only limited by the claims after authorization and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. In addition, the steps may be performed in an order different from that described in this disclosure. Further, the various elements in the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced by equivalent elements that appear after this disclosure.

Claims

1. A method for processing a traffic light image, comprising: Acquire a first image of the traffic light and perform a first thresholding process on the first image to obtain at least one first centroid; Acquire a first image set, the first image set including at least a second image taken after the first image; performing the first thresholding process on the second image to obtain at least one second centroid; for each first centroid of the at least one first centroid, in response to determining that the first centroid corresponds to one of the at least one second centroid, determining the first centroid as a candidate centroid; as well as Acquire a lamp head mass center corresponding to the lamp head of the signal lamp from one or more candidate mass centers among the at least one first mass center; Wherein, for each first center of mass of the at least one first center of mass, determining that the first center of mass corresponds to one of the at least one second center of mass includes: Obtaining at least one first position corresponding one-to-one to the at least one first center of mass and at least one second position corresponding one-to-one to the at least one second center of mass; For each first centroid of the at least one first centroid: Obtaining at least one distance from the first centroid to the at least one second centroid; and In response to determining that a distance from the first centroid to one of the at least one second centroid is less than a preset distance threshold, the first centroid is determined to correspond to the second centroid.

2. The method according to claim 1, wherein The performing a first thresholding process on the first image to obtain at least one first centroid comprises: Based on a first preset threshold, binarizing the first image to obtain a binarized image, wherein the first preset threshold corresponds to the color of the lamp head when it is lit; and Based on the binarized image, the at least one first centroid is obtained.

3. The method according to claim 1, wherein Each image in the first image set is taken within a preset time range after the first image, and wherein the preset time range corresponds to the duration of time when the lamp head is lit.

4. The method according to claim 1, wherein The step of obtaining the lamp head mass center corresponding to the lamp head of the signal lamp from the one or more candidate mass centers includes: determining a number of the one or more candidate centroids; In response to determining that the number of the one or more candidate centroids is equal to 1, determining the one or more candidate centroids to be the lamp head centroid; and In response to determining that the number of the one or more candidate centroids is greater than 1, a third image is acquired, and the lamp head centroid is acquired based on the third image.

5. The method according to claim 4, wherein The shooting time of the third image is separated from the shooting time of the first image by a preset time interval, the preset time interval being at least greater than the duration of the lamp head being lit, and obtaining the centroid of the lamp head based on the third image includes: performing differential processing on the first image and the third image to obtain a differential image; performing a second thresholding process on the difference image to obtain at least one difference centroid of the difference image; and In response to determining that one of the at least one differential centroid corresponds to one of the one or more candidate centroids, the one candidate centroid is determined to be the lamp head centroid.

6. The method according to claim 1, further comprising: The position of the lamp head is acquired based on the lamp head center of mass and a corresponding center of mass among the at least one second center of mass that corresponds to the lamp head center of mass.

7. A traffic light image processing device, comprising: a first acquiring unit configured to acquire a first image of the traffic light and perform a first thresholding process on the first image to obtain at least one first centroid; as well as a second acquiring unit configured to acquire a first image set, wherein the first image set at least includes a second image captured after the first image; a first thresholding processing unit configured to perform the first thresholding processing on the second image to obtain at least one second centroid; a determining unit configured to, for each first centroid of the at least one first centroid, determine the first centroid as a candidate centroid in response to determining that the first centroid corresponds to one of the at least one second centroid; as well as a lamp head mass center obtaining unit, configured to obtain a lamp head mass center corresponding to the lamp head of the signal lamp from one or more candidate mass centers among the at least one first mass center; Wherein, the determining unit includes: a third acquiring unit, configured to acquire at least one first position corresponding one-to-one to the at least one first mass center and at least one second position corresponding one-to-one to the at least one second mass center; a fourth acquiring unit configured to acquire, for each of the at least one first center of mass, at least one distance from the first center of mass to the at least one second center of mass; and A determination subunit is configured to determine, for each first center of mass among the at least one first center of mass, that the first center of mass corresponds to the second center of mass in response to determining that the distance from the first center of mass to a second center of mass among the at least one second center of mass is less than a preset distance threshold.

8. The device according to claim 7, wherein The first acquiring unit further includes: A binarization unit is configured to perform binarization processing on the first image based on a first preset threshold value to obtain a binarized image, wherein the first preset threshold value corresponds to the color of the lamp head when it is lit; and The first acquisition subunit is configured to obtain the at least one first centroid based on the binarized image.

9. The device according to claim 7, wherein The first image set and the second image are taken within a preset time range after the first image, and wherein the preset time range corresponds to the duration when the lamp head is lit.

10. The device according to claim 9, wherein The lamp holder mass center acquisition unit includes: a number determining unit configured to determine the number of the one or more candidate centroids; a lamp head centroid determining subunit, configured to, in response to determining that the number of the one or more candidate centroids is equal to 1, determine the one or more candidate centroids as the lamp head centroid; and The third image acquisition unit is configured to acquire a third image in response to determining that the number of the one or more candidate centroids is greater than 1, and acquire the lamp head centroid based on the third image.

11. The device according to claim 10, wherein The shooting time of the third image is separated from the shooting time of the first image by a preset time interval, and the preset time interval is at least greater than the duration of the lighting of the lamp head, and the third image acquisition unit includes: a differential processing unit configured to perform differential processing on the first image and the third image to obtain a differential image; a second thresholding processing unit configured to perform a second thresholding process on the difference image to obtain at least one candidate difference centroid of the difference image; and The matching unit is configured to, in response to determining that a differential centroid of the at least one differential centroid corresponds to a candidate centroid of the one or more candidate centroids, determine the candidate centroid as the lamp head centroid.

12. The device according to claim 9, wherein Also includes: The lamp head position acquisition unit is configured to acquire the position of the lamp head based on the lamp head center of mass and a corresponding center of mass in the at least one second center of mass corresponding to the lamp head center of mass.

13. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

15. A computer program product comprising a computer program, wherein When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

16. An edge computing device, comprising the electronic device according to claim 13.

Images