Image stripe detection methods, readable storage media, software products, and electronic devices

Abstract

This application relates to the field of assisted driving technology, and particularly to an image stripe detection method, a readable storage medium, a program product, and an electronic device. The image stripe detection method acquires a target image collected by a sensor and then determines the brightness parameters and extension direction of at least one stripe present in the target image. Based on the brightness parameters and extension direction of the at least one stripe, the map influence degree of the at least one stripe is determined, where the map influence degree represents the impact of the at least one stripe on the map accuracy when generating a map based on the target image. Through this method, an electronic device can detect the degree of influence of stripes on the target image on the generated map, thereby determining whether the sensor that acquired the target image is qualified, or whether the target image can be used to generate a map, to ensure the accuracy of the map generated by the vehicle based on the target image.

Description

Technical Field

[0001] This application relates to the field of driver assistance technology, and in particular to an image stripe detection method, a readable storage medium, a program product, and an electronic device. Background Technology

[0002] While a vehicle is in motion, its electronic devices (such as in-vehicle infotainment systems and onboard terminals) can use sensors to collect images of the surrounding environment and generate maps based on these images to assist driving. For example, a vehicle can use environmental images to identify road features such as lane lines or the center line of the road in its current lane, and then help the vehicle stay within the current lane.

[0003] However, environmental images acquired by vehicle sensors may contain stripe noise interference. For example, uneven response of sensors to different lighting conditions, or uneven power supply voltage, can lead to stripe noise interference in the environmental images. This stripe noise interference from sensors can affect the accuracy of maps generated by electronic devices based on environmental images, impacting the assisted driving capabilities of these devices and reducing the user's driving experience. Summary of the Invention

[0004] This application provides an image stripe detection method, a readable storage medium, a program product, and an electronic device. This method can detect the degree of influence of a target image on the vehicle's generated map, thereby detecting the vehicle's sensors and filtering out unqualified sensors or images, ensuring the accuracy of the map generated by the vehicle based on images acquired by the sensors.

[0005] In a first aspect, embodiments of this application provide an image stripe detection method applied to an electronic device. The method includes: acquiring a target image collected by a sensor; determining the brightness parameters and extension direction of at least one stripe present in the target image; and determining the map influence degree of the at least one stripe based on its brightness parameters and extension direction, wherein the map influence degree represents the impact of the at least one stripe on the map accuracy when a map is generated based on the target image.

[0006] In this method, the electronic device can detect the impact of abnormal stripes in the target image acquired by the sensor on the generated map. The electronic device can then determine whether to generate a map based on the target image, or whether to remove stripe noise from the target image, to ensure the accuracy of the generated map and improve the electronic device's assisted driving capabilities. Alternatively, during the detection of sensors to be installed on the vehicle, the electronic device can determine the sensor's quality by detecting the impact of abnormal stripes in the target image captured by the sensor on the generated map, thereby filtering out substandard sensors to ensure the vehicle's assisted driving capabilities and improve driving safety.

[0007] In one possible implementation of the first aspect above, determining the brightness parameter and extension direction of at least one stripe in the target image includes: determining the brightness parameter of a first type of stripe extending along a preset direction in the target image, and determining the extension direction of a preset second type of stripe corresponding to the target image.

[0008] In this possible implementation, the electronic device can determine the brightness parameters and extension direction of the stripes on the target image, respectively, in order to analyze the stripes on the target image and more accurately determine the degree of influence of the stripes on the target image on the generated map.

[0009] In one possible implementation of the first aspect described above, determining the brightness parameters of the first type of stripes extending along a preset direction in the target image includes: determining a plurality of image stripes from the target image, wherein each image stripe is determined based on at least one row of pixels arranged in a preset direction on the target image. Determining the first type of stripe among at least one stripe based on the brightness values ​​of the plurality of image stripes, and determining the brightness parameters based on the brightness values ​​of the first type of stripes, wherein the brightness value is the average of the pixel values ​​of the pixels on the corresponding stripe. In another possible implementation of the first aspect described above, determining the first type of stripe among at least one stripe based on the brightness values ​​of the plurality of image stripes includes: selecting a preset number of image stripes with the highest brightness values ​​from the plurality of image stripes as the first type of stripes.

[0010] In this possible implementation, the image stripes with the highest brightness may affect the generated map. Therefore, a preset number of image stripes with the highest brightness values ​​can be selected as the first type of stripes.

[0011] In one possible implementation of the first aspect described above, determining the brightness parameter based on the brightness value of the first type of stripes includes: determining the brightness parameter based on the ratio of the average brightness value of the first type of stripes to the average brightness value of the target image.

[0012] In one possible implementation of the first aspect above, determining the extension direction of the preset second type of stripe corresponding to the target image includes: converting the target image into a frequency domain spectrum based on the pixel values ​​of each pixel point on the target image; determining the position of the target point with the highest spectral value in the frequency domain spectrum, and determining the extension direction based on the position of the target point in the frequency domain spectrum, wherein the feature corresponding to the target point is the preset second type of stripe on the target image, and the second type of stripe is a stripe among at least one stripe.

[0013] In one possible implementation of the first aspect above, determining the extension direction based on the position of the target point in the frequency domain spectrum includes: determining a first straight line between the target position and the position of the zero frequency point in the frequency domain spectrum, and determining the extension direction based on a first angle between the first straight line and a first coordinate axis in the frequency domain spectrum.

[0014] In this possible implementation, the target point is the point with the largest spectral value on the frequency domain spectrum. That is, the feature corresponding to this spectral value has the most energy on the target image, occupying a significant amount of energy. Therefore, this feature is more prominent on the target image and can be considered as a second type of fringe. Furthermore, based on the position of the target point and the position of the zero-frequency, the first angle between the extension direction of the second type of fringe and the first coordinate axis (e.g., the width axis or the height axis) of the frequency domain spectrum can be determined. This allows for the determination of the extension direction of the second type of fringe.

[0015] In one possible implementation of the first aspect described above, determining the map influence of at least one stripe based on its brightness parameters and extension direction includes: determining the image quality of the target image based on the brightness parameters, where the image quality indicates the influence of the brightness of the first type of stripe on the image; determining the risk level of the target image to the generated map based on the extension direction, where the risk level indicates the influence of the distribution of the second type of stripe on the accuracy of road features on the map; and determining the map influence based on the image quality and the risk level.

[0016] In this possible implementation, the electronic device can fuse parameters such as brightness and extension direction to determine the map influence of at least one stripe on the target image based on the fusion result.

[0017] In one possible implementation of the first aspect described above, determining the image quality of the target image based on brightness parameters includes: determining a relative standard deviation based on the standard deviation and mean of the pixel values ​​corresponding to the pixels of the target image; and determining an amplitude ratio based on the spectral values ​​of the target points and the median of the spectral values ​​in the frequency domain spectrogram. The brightness parameters are mapped to a first score, the relative standard deviation to a second score, and the amplitude ratio to a third score. A target score is determined based on the weighted sum of the first, second, and third scores, and the image quality is determined according to the magnitude of the target score.

[0018] In this possible implementation, the electronic device can determine the quality of the target image based on parameters such as brightness parameters, relative standard deviation, and amplitude ratio, and then determine the impact of the target image on the generated map based on the quality of the target image.

[0019] In one possible implementation of the first aspect above, determining the risk level of the target image to the generated map based on the extension direction includes: determining the spatial frequency based on the distance between the target point and the zero-frequency point of the frequency domain spectrum, the spatial frequency being used to indicate the density of the second type of stripes distributed on the target image; and determining the risk level based on the spatial frequency and the extension direction.

[0020] In this possible implementation, the electronic device can determine the influence of at least one stripe on the target image on road elements (such as lane lines, road center lines, etc.) on the vehicle-generated map based on parameters such as the extension direction and spatial frequency of at least one stripe on the target image, thereby judging the risk level of map generation based on the target image, so as to ensure the detection accuracy of the map influence of the electronic device on the target image.

[0021] Secondly, this application provides an electronic device, comprising: a memory for storing instructions; and at least one processor for executing the instructions to cause the electronic device to implement the vehicle speed determination method provided in the first aspect and any possible implementation of the first aspect. The beneficial effects achievable in the second aspect can be referred to the beneficial effects of the method provided in any embodiment of the first aspect, and will not be repeated here.

[0022] Thirdly, this application provides a vehicle including the electronic equipment described in the second aspect. The beneficial effects achievable through this third aspect are similar to those of the electronic equipment provided in the second aspect, and will not be repeated here.

[0023] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to implement the vehicle speed determination method provided in the first aspect and any possible implementation of the first aspect. The beneficial effects achievable in the fourth aspect can be found in the beneficial effects of the method provided in any embodiment of the first aspect, and will not be repeated here.

[0024] Fifthly, this application provides a computer program product that stores instructions that, when executed on a device, cause the device to implement the vehicle speed determination method provided in the first aspect and any possible implementation of the first aspect. The beneficial effects achievable in the fifth aspect can be found in the beneficial effects of the method provided in any embodiment of the first aspect, and will not be repeated here. Attached Figure Description

[0025] Figure 1A A schematic diagram of a vehicle driving on a road is shown;

[0026] Figure 1B A schematic diagram of the surrounding environment captured by a vehicle is shown;

[0027] Figure 1C A schematic diagram of a vehicle-generated map is shown;

[0028] Figure 2 According to some embodiments of this application, a flowchart of an image stripe detection method is shown;

[0029] Figure 3A According to some embodiments of this application, a schematic diagram of the brightness value of an image is shown;

[0030] Figure 3B A schematic diagram of a frequency domain spectrum is shown according to some embodiments of this application;

[0031] Figure 4 According to some embodiments of this application, a structural schematic diagram of a vehicle is shown. Detailed Implementation

[0032] The illustrative embodiments of this application include, but are not limited to, image stripe detection methods, readable storage media, program products, and electronic devices.

[0033] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0034] As shown in the background section, noise interference may exist when sensors acquire images of the environment around a vehicle. This noise interference affects the accuracy of the maps generated by electronic devices and the assisted driving capabilities of these devices.

[0035] For example, Figure 1A A schematic diagram of a vehicle driving on a road is shown; Figure 1B A schematic diagram of the surrounding environment captured by a vehicle is shown. Figure 1C A schematic diagram of a vehicle-generated map is shown.

[0036] Reference Figures 1A to 1C Vehicle 01 is driving on a road, and the road features include lane lines C1. Sensors on vehicle 01 (such as cameras) can collect environmental images P1 around vehicle 01, and then electronic devices configured on vehicle 01 can generate a map T1 based on the environmental images P1.

[0037] However, the sensors on vehicle 01 may be affected by electromagnetic interference or unstable power supply voltage, resulting in stripe noise P11 in the acquired environmental image P1. If the electronic equipment on the vehicle generates map T1 from the environmental image P1, the lane lines C1 on map T1 may be affected by the stripe noise P11, causing the lane lines C1 on the map T1 generated by the electronic equipment to be skewed, thereby affecting the assisted driving capabilities of the electronic equipment.

[0038] To address the aforementioned problems, this application provides an image stripe detection method. An electronic device acquires a target image collected by a sensor and determines the brightness parameters and extension direction of at least one stripe present in the target image. Then, based on the brightness parameters and extension direction of the at least one stripe, the map influence degree of the at least one stripe is determined. The map influence degree represents the impact of the at least one stripe on the map accuracy when generating a map based on the target image.

[0039] Through the above scheme, during vehicle operation, electronic devices can detect the impact of abnormal stripes in the target image acquired by the sensor on the generated map. The electronic devices can then determine whether to generate a map based on the target image, or whether stripe noise removal is necessary, to ensure the accuracy of the generated map and improve the vehicle's assisted driving capabilities. Alternatively, during the testing of sensors to be installed on the vehicle, the impact of abnormal stripes in the target image captured by the sensor on the generated map can be detected to determine the sensor's quality, thereby filtering out substandard sensors to ensure the vehicle's assisted driving capabilities and improve driving safety.

[0040] In some embodiments of this application, the sensor can be a camera, which may include a front-view camera, a rear-view camera, a side-view camera, or a surround-view camera, etc. The target image can be an environmental image of the vehicle's surroundings collected by sensors configured on the vehicle, or a test image captured during quality inspection of the sensor. For example, the sensor can be masked to capture a pure black target image, and then stripe detection can be performed based on the pure black target image. In this way, when stripe interference exists in the sensor, the stripes are more obvious on the pure black image, thereby improving the accuracy of image stripe detection.

[0041] The image stripe detection method in the embodiments of this application is described below.

[0042] For example, Figure 2 According to some embodiments of this application, a flowchart of an image stripe detection method is shown.

[0043] It is understood that the following processes can be performed by electronic devices, such as vehicle infotainment systems, smart cockpits, controllers, in-vehicle tablets, in-vehicle computers, etc., or any electronic device with computing capabilities, such as a server, used to test sensors. Below, taking a vehicle infotainment system as an example, the image stripe detection method in this application embodiment is described.

[0044] like Figure 2 As shown, the process includes:

[0045] S201, acquire the target image collected by the sensor, and determine the brightness parameters and extension direction of at least one stripe in the target image.

[0046] For example, in some embodiments of this application, the electronic device can acquire a target image collected by a sensor. The electronic device can then detect the brightness parameters and extension direction of at least one stripe present in the target image.

[0047] In some embodiments of this application, the electronic device can preprocess the target image. For example, after acquiring the target image (e.g., an image supporting RGB / grayscale format), the electronic device can first determine whether the image size exceeds a size threshold, where the size threshold can be a pixel value between 1000 and 3000. In some embodiments of this application, the size threshold is 2000 pixels. The electronic device can also determine whether the width and height of the target image exceed 2000 pixels. If either side exceeds 2000 pixels, a bilinear interpolation algorithm is used to automatically reduce the target image proportionally, thus improving the speed at which the electronic device processes the target image. The electronic device can also convert the target image into a grayscale image, thereby determining abnormal stripes in the target image based on the grayscale values ​​of each pixel (as an example of pixel values). In other embodiments, when detecting whether a sensor is qualified, the electronic device can also retain the target image for result display.

[0048] In some embodiments of this application, the electronic device can determine data such as brightness parameters of an image from the temporal domain of the image. The temporal domain of the image refers to its original spatial representation, i.e., the variation of pixel grayscale or color values ​​with spatial coordinates. For example, in the temporal domain of a target image, the electronic device can determine multiple image stripes from the target image. Each image stripe is determined based on at least one row of pixels arranged in a preset direction on the target image. Then, the electronic device determines a first type of stripe from at least one stripe based on the brightness values ​​of the multiple image stripes, and determines the brightness parameters based on the brightness values ​​of the first type of stripe, where the brightness value is the average pixel value of the corresponding pixel on the stripe.

[0049] Furthermore, the electronic device can convert the target image into a frequency domain spectrum based on the pixel values ​​of each pixel on the target image, determine the position of the target point with the highest spectral value in the frequency domain spectrum, and determine the extension direction based on the position of the target point in the frequency domain spectrum. The extension direction indicates that the target image includes a second type of stripe extending along the extension direction, and the second type of stripe is a stripe in at least one stripe.

[0050] For example, in embodiments of this application, the preset direction can be the width and / or height direction in the coordinate system of the target image. It is understood that during vehicle travel, road elements (such as lane lines, road center lines, stop lines, or zebra crossings) generally extend along the width or height direction of the target image. Therefore, the width and / or height direction in the coordinate system of the target image can be used as the preset direction. In other embodiments, the preset direction can also be set as needed. For example, the preset direction can also be a direction in the target image coordinate system that makes a certain angle with the width direction. Embodiments of this application do not limit the preset direction.

[0051] For example, in some embodiments of this application, the preset direction is taken as the width direction. The electronic device can use a row of pixels arranged along the width direction of the target image as an image stripe. That is, if the target image includes 1,000 rows of pixels, then the target image will have 1,000 image stripes.

[0052] Then, the electronic device selects a preset number of image stripes with the highest brightness values ​​from multiple image stripes as the first type of stripes. In some embodiments of this application, the preset number can be any value from 5% to 20% of the current row number of the target image. Preferably, the preset number can be 10% of the current row number of the target image. For example, in an embodiment of this application, the size of the target image can be 2000×1000. That is, the width direction of the target image includes 2000 pixels and the height direction includes 1000 pixels. When the preset direction is the width direction, there are a total of 1000 image stripes, and each image stripe includes 2000 pixels. The brightness value of each image stripe can be the average value of the 2000 pixels of the corresponding image stripe. For example, in some embodiments of this application, after the target image is converted into a grayscale image, the pixel value of each pixel can be a value between [0, 255], where the larger the pixel value, the brighter the corresponding pixel. That is, a pixel value of 0 is black, and a pixel value of 255 is white.

[0053] After determining the brightness values ​​of multiple image stripes, the image stripes with the highest brightness values ​​in the top 10% are designated as the first category of stripes. For example, after determining the brightness values ​​of 1000 image stripes, the image stripes with the highest brightness values ​​can be designated as the first category of stripes.

[0054] As an example, Figure 3A According to some embodiments of this application, a schematic diagram of the brightness value of an image is shown.

[0055] like Figure 3A As shown, the average brightness of the pixels in the target image is approximately 17.6 (e.g., ...). Figure 3A As shown by the dashed lines in the image, the target image comprises 1600 pixels in its height direction, corresponding to 1600 rows of pixels (each row of pixels corresponds to one image stripe). The electronic device can then determine the 160 rows with the highest brightness values ​​from these 1600 rows as the first type of stripes and determine the brightness parameters based on the ratio of the average brightness value of the first type of stripes to the average brightness value of all pixels in the target image. The average brightness value of the first type of stripes can be the average of the pixel values ​​of all pixels included in the first type of stripes.

[0056] For example, the electronic device determines that the average brightness value of the first type of stripes is μstripe And determine the global average brightness μ of the target image. global Then, the brightness and darkness parameters can be determined according to the following equation (1):

[0057] R brightness =μ stripe / μ global (1)

[0058] Among them, R brightness This is a brightness parameter, which in some embodiments may also be referred to as the stripe brightness ratio. stripe The average value of the brightness of the first type of stripes, μ global This represents the average brightness of all pixels in the target image.

[0059] The degree of influence of the first type of stripes on the target image can be determined by the brightness parameter of the first type of stripes. The larger the brightness parameter, the more obvious the first type of stripes are. If a map is generated based on the target image, the first type of stripes will have a greater impact on road elements extending / arranging along a preset direction in the map, and the map generated by the electronic device based on the target image will be more affected by the first type of stripes.

[0060] In some embodiments of this application, the electronic device can also convert the target image to the frequency domain, thereby analyzing the frequency domain of the target image to determine the extension direction of a preset second type of fringe corresponding to the target image. The frequency domain is the representation obtained after the image undergoes a Fourier Transform, decomposing the image into a superposition of sine and cosine waves of different frequencies. Furthermore, the electronic device can convert the target image into a frequency domain spectrum through a Fourier transform, and then determine the extension direction of the second type of fringe based on the target point with the largest spectral value in the frequency domain spectrum. It can be understood that the feature corresponding to the target point is a preset second type of fringe. For example, after converting the target image to a frequency domain spectrum, a first straight line between the target point and the position of the zero-frequency point in the frequency domain spectrum can be determined, and then the extension direction can be determined based on the first angle between the first straight line and the first coordinate axis in the frequency domain spectrum.

[0061] For example, after converting the target image into a grayscale image, the electronic device can perform a Fast Fourier Transform (FFT) on the grayscale image to convert the pixel values ​​of the grayscale image to the frequency domain, thereby generating a frequency domain spectrum. For example, the formula for the Fast Fourier Transform can be found in the following equation (2):

[0062] (2)

[0063] In this context, F(u,v) is the spectral value corresponding to coordinates (u,v) in the frequency domain spectrum. u represents the spatial frequency in the width (or horizontal) direction of the target image, measured in periods per image width. It describes the rate of grayscale change in the target image along the width direction. v represents the spatial frequency in the height (or vertical) direction of the target image, measured in periods per image height. It describes the rate of grayscale change in the target image along the height direction. F(u,v) is the complex representation of the image in the frequency domain, corresponding to the combined effect of u and v. F(u,v) represents the intensity and phase of the components in the target image that simultaneously possess u and v. |F(u,v)| can represent the energy or intensity of the frequency component (u,v), i.e., the degree of "distribution" of that frequency in the target image and the degree of pixel value change. For example, if a certain feature in the target image has a frequency u changing along the width direction and a frequency v changing along the height direction, and the pixel value of that feature is large, then |F(u,v)| will be large. f(x,y) represents the pixel value of the pixel at coordinates (x,y) in the target image. x is the coordinate in the width direction of the target image, y is the coordinate in the height direction of the target image, M is the width of the target image, N is the height of the target image, and j is the imaginary unit.

[0064] In some embodiments of this application, after determining the spectral value F(u,v) of the corresponding point in the frequency domain spectrum, the logarithm of the spectral value F(u,v) can be taken to improve the visualization effect of the frequency domain spectrum. For example, the process of taking the logarithm of F(u,v) can refer to the following equation (3):

[0065] S(u,v)=log 10 (1+| F(u,v)|) (3)

[0066] Where S(u,v) is the logarithmic value of the spectrum corresponding to coordinate (u,v) in the frequency domain spectrum, and F(u,v) is the spectrum value corresponding to coordinate (u,v). |A| represents taking the absolute value of A.

[0067] In some embodiments of this application, after determining S(u,v), the zero-frequency point (the point where both u and v are 0) can be moved to the center position to generate a frequency domain spectrum. It can be understood that moving the zero-frequency point to the center of the frequency domain spectrum can more intuitively represent the frequency distribution in the frequency domain spectrum.

[0068] As an example, Figure 3B According to some embodiments of this application, a schematic diagram of a frequency domain spectrum is shown.

[0069] refer to Figure 3BAfter moving the zero-frequency point to the center, the central region of the frequency domain spectrum is generally composed of low-frequency coordinate points. This central region can be a region with a radius of r, where r is 1 / 10 of the minimum of the width and height of the frequency domain spectrum. It can be understood that after moving the zero-frequency point of the frequency domain spectrum to the center, lower-frequency coordinate points are closer to the center of the spectrum. These low-frequency coordinate points correspond to pixels in the normal region of the target image; the normal region refers to an area with little or no noise interference. Therefore, in the embodiments of this application, the coordinate points in the central region of the frequency domain spectrum can be removed first, thereby avoiding the influence of the coordinate points in the central region on the direction of extension of the second type of fringes.

[0070] In some embodiments of this application, after removing the central region of the frequency domain spectrum, the electronic device can determine the point with the highest spectral value from the remaining coordinates of the frequency domain spectrum as the target point. The coordinate point with the highest spectral value in the frequency domain spectrum is typically directly related to the second type of stripes present in the corresponding target image.

[0071] It is understandable that the essence of stripe interference is the existence of periodic or quasi-periodic spatial patterns in the image, and the Fourier transform converts this periodicity into discrete frequency peaks in the frequency domain. Furthermore, the extension direction of the stripes is related to the distribution of the target points. For example, if the target points are distributed on the width axis of the frequency domain spectrum, it indicates that the second type of stripes have a higher frequency distribution along the width direction of the target image, and the extension direction of the second type of stripes is perpendicular to the width of the target image (i.e., perpendicular to the distribution direction). Therefore, in some embodiments of this application, the first angle between the line connecting the target point and the origin of the frequency domain spectrum and the first coordinate axis in the frequency domain spectrum can be determined based on the position of the target point in the frequency domain spectrum. It is understood that the origin of the frequency domain spectrum is the zero-frequency point; after the zero-frequency point is transformed to the center position of the frequency domain spectrum, the origin position is located at the center of the frequency domain spectrum. The first coordinate axis can be the width axis of the frequency domain spectrum (corresponding to the U-axis), and the first angle can be determined based on the line connecting the target point and the origin and the first coordinate axis. For example, if the first angle is 30°, then the direction in which the abnormal stripes extend may be a direction with an angle of 150° to the width direction of the frequency domain spectrum.

[0072] The first angle can be determined according to the following equation (4):

[0073] θ=arctan(Δu / Δv)×(180° / π) (4)

[0074] Where θ is the first angle, Δu is the coordinate difference between the target point and the zero frequency point in the width direction (hereinafter referred to as the U direction) of the frequency domain spectrum, and Δv is the coordinate difference between the target point and the zero frequency point in the height direction (hereinafter referred to as the V direction) of the frequency domain spectrum. The distribution direction of the abnormal stripes can be determined by equation (4), and then the extension direction of the abnormal stripes can be determined. That is, the extension direction is the direction with an angle of 180°-θ with the width direction.

[0075] S202, based on the brightness parameters and extension direction of at least one stripe, determine the map influence of at least one stripe, where the map influence represents the influence of at least one stripe on the map accuracy when generating a map based on the target image.

[0076] Exemplary examples, in some embodiments of this application, after the electronic device determines the brightness parameters and extension direction of at least one stripe, it can determine the image quality of the target image based on the brightness parameters, whereby the image quality indicates the influence of the brightness of the first type of stripes on the image. Furthermore, it can determine the risk level of the target image to the generated map based on the extension direction, whereby the risk level indicates the influence of the distribution of the second type of stripes on the accuracy of road features on the map. Then, the electronic device can determine the map impact level based on the image quality and the risk level.

[0077] For example, the electronic device can also determine the relative standard deviation based on the standard deviation and mean of the pixel values ​​corresponding to the pixels of the target image, and determine the amplitude ratio based on the spectral values ​​of the target points and the median of the spectral values ​​in the frequency domain spectrogram. The electronic device then maps the brightness parameter to a first score, the relative standard deviation to a second score, and the amplitude ratio to a third score, and determines the target score based on the weighted sum of the first, second, and third scores. Finally, the image quality is determined based on the magnitude of the target score.

[0078] The amplitude ratio indicates the ratio of the peak noise intensity of the second type of fringes to the background noise intensity; a larger amplitude ratio indicates more pronounced noise in the second type of fringes. The relative standard deviation indicates the noise level of the image; a larger relative standard deviation suggests potentially greater noise.

[0079] For example, in some embodiments of this application, the amplitude ratio can be determined according to the following equation (5):

[0080] R amplitude =P peak / M median (5)

[0081] Among them, R amplitude For amplitude ratio, P peak M represents the spectral value of the target point. medianThis is the median of the spectral values ​​at each coordinate point in the frequency domain spectrogram after removing the central region. In other words, frequency domain analysis can also determine parameters such as the amplitude ratio of the image.

[0082] The relative standard deviation is determined according to the following equation (6):

[0083] C stripe =σ row / μ global (6)

[0084] Among them, C stripe σ represents the relative standard deviation. row μ is the standard deviation of the pixel values ​​in the target image. global This represents the average pixel value of the target image.

[0085] In some embodiments of this application, when determining image quality, the electronic device can perform parameter fusion of the brightness parameters, amplitude ratio, and relative standard deviation determined above.

[0086] For example, in some embodiments, a piecewise linear interpolation function can be used to map the amplitude ratio, brightness parameter, and relative standard deviation to scores (e.g., full scores of 30, 40, and 30 points respectively). For example, the brightness parameter R... brightness The scoring function is:

[0087] If R brightness If ≤ 1.12, then the score for the brightness parameter is 40 (as an example of the first score).

[0088] If 1.12 <R brightness If ≤ 1.20, then the score for the brightness parameter = 35 + 5 × (1.20 - R) brightness ) / 0.08.

[0089] If 1.04 <R brightness If ≤ 1.12, then the score for the brightness parameter = 30 + 5 × (1.12 - R) brightness ) / 0.08.

[0090] ...

[0091] If 0 <R brightness If the value is ≤0.08, then the score for the brightness parameter = 5 × (0.08 - R). brightness ) / 0.08.

[0092] Similarly, the amplitude ratio and relative standard deviation are also mapped to fractions using a piecewise linear interpolation function. In some embodiments of this application, thresholds for the piecewise functions of the aforementioned brightness parameters, amplitude ratio, and relative standard deviation can be established through in-vehicle image analysis experiments, based on classical theory, experimental data, and practical engineering experience. Furthermore, these thresholds can be adjusted according to different application scenarios and security levels; they are not absolutely fixed and can be calibrated in conjunction with the characteristics of specific products, usage environments, and safety requirements.

[0093] After determining the amplitude ratio, brightness parameters, and relative standard deviation of the target image, scores can be determined for each of these parameters. These three scores are then weighted and summed (e.g., weights of 30%, 40%, and 30% respectively) to obtain the target score (total score). This process fuses the brightness parameters, amplitude ratio, and relative standard deviation. The electronic device can then determine the image quality corresponding to the target score based on a preset quality level.

[0094] For example, preset quality levels can include: Excellent (target score greater than or equal to 85 points), Good (target score any value between 70 and 84 points), Average (target score any value between 50 and 69 points), and Poor (target score less than 50 points). The electronic device can then determine the image quality of the target image based on its target score.

[0095] In some embodiments of this application, after the electronic device determines the target point, it can also determine the spatial frequency based on the distance between the target point and the zero-frequency point of the frequency domain spectrum. The spatial frequency is used to indicate the density of the second type of stripes distributed on the target image. Furthermore, the risk level is determined based on the spatial frequency and the direction of extension. A higher spatial frequency indicates a higher frequency of the second type of stripes. Thus, the distribution of the second type of stripes can be determined based on their spatial frequency, thereby determining the impact of the second type of stripes in the target image on road features on the generated map.

[0096] For example, the spatial frequency can be determined according to the following equation (7):

[0097] F spatial =sqrt(Δu 2 +Δv 2 ) / min(M,N) (7)

[0098] Among them, F spatialLet Δu be the spatial frequency, Δv be the coordinate difference between the target point and the zero-frequency point in the U direction of the frequency domain spectrum, sqrt(A) be used to take the square root of A, min(A, B) be used to take the minimum value of A and B, M be the width of the target image, and N be the height of the target image. It can be understood that the spatial frequency determined by equation (7) is the normalized distance between the target point and the origin. It can be understood that the origin is the zero-frequency point; if the zero-frequency point of the frequency domain spectrum is moved to the center position, then the spatial frequency is the normalized distance between the target point and the center point.

[0099] In some embodiments of this application, after determining the first angle and spatial frequency, the impact of anomalous stripes on the risk level of road features on the generated map can be determined.

[0100] For example, if 85°≤|θ|≤ 95°, then mark it as "Horizontal stripe risk: may interfere with lane line detection".

[0101] If 0.10≤F spatial ≤0.25, marked "Frequency overlap risk: Noise frequency overlaps with road marking frequency".

[0102] It is understandable that the map impact of a target image can be determined by considering the image quality and risk level mentioned above. For example, if the image quality level is "excellent" and the risk level is "no special risk markers", then the map impact is "low risk - suitable for all vehicle applications".

[0103] If the quality rating is "Good" but there is a risk of horizontal stripes, the conclusion is "Medium risk - horizontal stripes may interfere with lane line detection".

[0104] Furthermore, electronic devices can determine whether to generate a map based on the target image's map influence.

[0105] It is understood that, in other embodiments, the electronic device can also determine the quality of the image captured by the sensor based on the map influence of the target image, thereby determining whether the sensor has defects and detecting the sensor.

[0106] Through the above methods, electronic devices can detect the map impact of target images to determine whether the target image can be used to generate a map, thereby ensuring the accuracy of the map generated by the electronic device and improving its assisted driving capabilities. Alternatively, it can determine whether the sensors used to acquire target images are qualified, thus ensuring sensor quality and enabling the sensors to acquire accurate target images, improving driving safety.

[0107] The vehicles in the above embodiments will now be described.

[0108] For example, Figure 4 According to some embodiments of this application, a structural schematic diagram of a vehicle 01 is shown.

[0109] Understandable. Figure 4 This is a schematic diagram of a possible functional framework for a vehicle 01 provided in an embodiment of this application. As shown in Figure 4, the functional framework of the vehicle 01 may include various subsystems, such as a sensor system 10, a control system 20, and one or more peripheral devices 30. Figure 4 (Taking one as an example), power supply 40, computer system 50. Optionally, vehicle 01 may also include other functional systems, such as an engine system that provides power to vehicle 01, etc., which are not limited here. It is understood that the electronic devices in the embodiments of this application may be devices on vehicle 01 that include computer system 50.

[0110] The sensor system 10 may include several detection devices that can sense the measured information and convert the sensed information into electrical signals or other desired forms of information output according to a certain rule. For example... Figure 4 As shown, these detection devices may include a Global Positioning System (GPS) 11, a vehicle speed sensor 12, an Inertial Measurement Unit (IMU) 13, etc., and this application is not limited thereto. The GPS 11 is a system that uses GPS positioning satellites to perform real-time positioning and navigation globally. In this application, the vehicle speed sensor 12 is used to detect the vehicle speed of vehicle 01. The inertial measurement unit 13 may include a combination of an accelerometer and a gyroscope, and is a device for measuring the yaw rate and acceleration of vehicle 01. For example, during the movement of vehicle 01, the inertial measurement unit can measure the changes in the vehicle's position and angle based on the inertial acceleration of vehicle 01, such as measuring the acceleration and yaw rate of vehicle 01.

[0111] The control system 20 may include a steering unit 21, a braking unit 22, etc.

[0112] The steering unit 21 can represent a system for adjusting the direction of travel of vehicle 01, which may include, but is not limited to, a steering wheel or other structural devices for adjusting or controlling the direction of travel of vehicle 01. In embodiments of this application, vehicle 01 can determine data such as the steering wheel angle through the steering unit 21. The braking unit 22 can represent a system for slowing down the speed of vehicle 01, and may also be referred to as the vehicle 01 braking system. It may include, but is not limited to, a brake controller, a reducer, or other structural devices for slowing down vehicle 01. In practical applications, the braking unit 22 can use friction to slow down the tires of vehicle 01, thereby slowing down the speed of vehicle 01. For example, the vehicle's AEB system may include the braking unit 22, which can be controlled to brake when a collision with an obstacle is predicted.

[0113] Peripheral device 30 may include several components, such as Figure 4The system includes a communication system 31, a touchscreen system 32, a user interface 33, etc. The communication system 31 is used to enable network communication between the vehicle 01 and other devices, such as electronic devices. In practical applications, the communication system 31 can use wireless communication technology or wired communication technology to achieve network communication between the vehicle 01 and other devices. This wired communication technology can refer to communication between the vehicle 01 and other devices via network cables or optical fibers. This wireless communication technology includes, but is not limited to, Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), Wireless Local Area Networks (WLAN) (such as Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR) technology, etc.

[0114] The touchscreen system 32 can be used to detect operation commands on the touchscreen. For example, a user can perform touch operations on the content data displayed on the touchscreen according to actual needs to achieve the corresponding function, such as playing music, video, or other multimedia files. The user interface 33 can specifically be a touch panel for detecting operation commands on the touch panel. The user interface 33 can also be a physical button or a mouse. The user interface 33 can also be a display screen for outputting data and displaying images or data. Optionally, the user interface 33 can also be at least one device belonging to the category of peripheral devices, such as a touchscreen, microphone, and speaker.

[0115] Several functions of vehicle 01 are controlled and implemented by computer system 50. Computer system 50 may include multiple processors such as a general-purpose processor 51, a continuous damping control system (CDC) 52, a mobile data center (MDC) 53, a telematics box (T-BOX) 54, as well as a memory 55 (also referred to as a storage device) and a gateway 56. In practical applications, the memory 55 may be located inside or outside the computer system 50, for example, as a cache within vehicle 01; this application does not impose limitations. The general-purpose processor 51 may be a graphics processing unit (GPU), etc. The general-purpose processor 51, CDC 52, MDC 53, and T-BOX 54 can be used to run relevant programs or corresponding instructions stored in memory 55 to implement the corresponding functions of vehicle 01, such as network switching functions based on service units.

[0116] The memory 55 may include volatile memory, such as RAM; it may also include non-volatile memory, such as ROM, flash memory, HDD, or SSD; or it may include a combination of the above types of memory. The memory 55 can be used to store a set of program code or instructions corresponding to the program code, so that the general-purpose processor 51 can call the program code or instructions stored in the memory 55 to implement the corresponding functions of the vehicle 01. This function includes, but is not limited to, […]. Figure 4 The schematic diagram of the functional framework of vehicle 01 shown includes some or all of the functions. In this application, memory 55 can store a set of program codes for controlling vehicle 01. The general-purpose processor 51, CDC 52, MDC 53, and T-BOX 54 can call this program code to control vehicle 01 to execute the trajectory prediction method in this application.

[0117] Optionally, in addition to storing program code or instructions, the memory 55 may also store information such as road maps, driving routes, and sensor data. The computer system 50 can be combined with other components in the functional framework diagram of the vehicle 01, such as sensors in the sensor system and GPS, to realize the relevant functions of the vehicle 01. For example, the computer system 50 can control the driving direction or speed of the vehicle 01 based on the data input from the sensor system 10; this application does not impose limitations on this.

[0118] This application also provides a program product that stores instructions. When these instructions are executed on an electronic device, they enable the electronic device to implement the methods provided in the foregoing embodiments.

[0119] This application also provides a readable storage medium storing one or more programs, which, when executed by an electronic device, enable the electronic device to implement the methods provided in the foregoing embodiments.

[0120] It should be noted that the above Figure 4 This is merely a schematic diagram of one possible functional framework for vehicle 01. In practical applications, vehicle 01 may include more or fewer systems or components, and this application is not limiting. Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0121] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor. The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used to implement the program code when necessary. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0122] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other propagation signals. Therefore, machine-readable media include any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0123] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0124] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0125] It should be noted that in the examples and description of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0126] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made thereto without departing from the scope of this application.

Claims

1. A method for detecting stripes in an image, characterized in that, Applied to electronic devices, the method includes: Acquire the target image captured by the sensor; Determine the brightness parameters and extension direction of at least one stripe in the target image; Based on the brightness parameters and extension direction of the at least one stripe, the map influence of the at least one stripe is determined, wherein the map influence represents the impact of the at least one stripe on the map accuracy when a map is generated based on the target image.

2. The image stripe detection method according to claim 1, characterized in that, Determining the brightness parameters and extension direction of at least one stripe in the target image includes: The brightness parameters of the first type of stripes extending along a preset direction in the target image are determined, and the extension direction of the preset second type of stripes corresponding to the target image is determined.

3. The image stripe detection method according to claim 2, characterized in that, The determination of the brightness parameters of the first type of stripes extending along a preset direction in the target image includes: Multiple image stripes are determined from the target image, wherein each image stripe is determined based on at least one row of pixels arranged in a preset direction on the target image; Based on the brightness values ​​of the plurality of image stripes, a first type of stripe is determined among the at least one stripe, and the brightness parameter is determined according to the brightness value of the first type of stripe, wherein the brightness value is the average value of the pixel values ​​of the corresponding stripe.

4. The image stripe detection method according to claim 3, characterized in that, Determining the first type of stripe among the at least one stripe based on the brightness values ​​of the plurality of image stripes includes: selecting a preset number of image stripes with the highest brightness values ​​from the plurality of image stripes as the first type of stripes.

5. The image stripe detection method according to claim 3, characterized in that, Determining the brightness parameter based on the brightness value of the first type of stripes includes: The brightness parameter is determined based on the ratio of the average brightness value of the first type of stripes to the average brightness value of the target image.

6. The image stripe detection method according to claim 2, characterized in that, Determining the extension direction of the preset second type of stripes corresponding to the target image includes: Based on the pixel values ​​of each pixel in the target image, the target image is converted into a frequency domain spectrogram; The position of the target point with the highest spectral value in the frequency domain spectrum is determined from the frequency domain spectrum, and the extension direction is determined based on the position of the target point in the frequency domain spectrum. The feature corresponding to the target point is a preset second type of stripe on the target image, and the second type of stripe is a stripe in at least one stripe.

7. The image stripe detection method according to claim 6, characterized in that, Determining the extension direction based on the position of the target point in the frequency domain spectrum includes: Determine the first straight line between the target point and the position of the zero frequency point in the frequency domain spectrum; The extension direction is determined based on the first angle between the first straight line and the first coordinate axis in the frequency domain spectrum.

8. The image stripe detection method according to claim 7, characterized in that, The determination of the map influence of the at least one stripe based on its brightness parameters and extension direction includes: The image quality of the target image is determined based on the brightness parameters, and the image quality is used to indicate the influence of the brightness of the first type of stripes on the image; The risk level of the target image to the generated map is determined based on the extension direction, and the risk level indicates the impact of the distribution of the second type of stripes on the accuracy of road features on the map; The map impact is determined based on the image quality and the degree of risk.

9. The image stripe detection method according to claim 8, characterized in that, Determining the image quality of the target image based on the brightness parameters includes: The relative standard deviation is determined based on the standard deviation and mean of the pixel values ​​corresponding to the pixels of the target image. Furthermore, the amplitude ratio is determined based on the spectral value of the target point and the median of the spectral values ​​in the frequency domain spectrum. The brightness parameter is mapped to a first score, the relative standard deviation is mapped to a second score, and the amplitude ratio is mapped to a third score. The target score is determined based on the weighted sum of the first score, the second score, and the third score; The image quality is determined based on the magnitude of the target score.

10. The image stripe detection method according to claim 8, characterized in that, The step of determining the risk level of the target image to the generated map based on the extension direction includes: Based on the distance between the target point and the zero-frequency point of the frequency domain spectrum, a spatial frequency is determined, which is used to indicate the density of the second type of stripes distributed on the target image. The degree of risk is determined based on the spatial frequency and the direction of extension.

11. An electronic device, characterized in that, Includes memory for storing instructions; At least one processor is configured to execute the instructions to cause the electronic device to implement the image stripe detection method according to any one of claims 1 to 10.

12. A vehicle, characterized in that, Includes the electronic device of claim 11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the image stripe detection method according to any one of claims 1 to 10.

14. A computer program product, characterized in that, When the computer program product is run on the device, it causes the device to perform the image stripe detection method according to any one of claims 1 to 10.

Images