Environmental visibility evaluation method, vehicle lamp control method, and vehicle‑mounted electronic device

Abstract

The present application relates to an environmental visibility evaluation method, a vehicle lamp control method, and a vehicle‑mounted electronic device. The method comprises: acquiring a real-time image captured by a vehicle-mounted camera, and an actual photographing distance of the camera; extracting a first stroke feature of a character in the real-time image and a second stroke feature of the character in a complete form; by using the second stroke feature as a reference object, determining a stroke continuity degree of the first stroke feature relative to the second stroke feature; and on the basis of the stroke continuity degree and the actual photographing distance, determining the visibility of a road environment at the moment the real-time image is captured.

Description

Environmental visibility assessment methods, vehicle lighting control methods, and onboard electronic devices Cross-reference to related applications

[0000] This application claims priority to Chinese Patent Application No. 202411821458.3, filed with the Chinese Patent Office on December 11, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0001] The embodiments of this application relate to, but are not limited to, the field of vehicle safe driving technology, and in particular to, but are not limited to, an environmental visibility assessment method, a vehicle lighting control method, and an in-vehicle electronic device. Background Technology

[0002] With the development of automotive electronics technology, vehicle safety features are becoming increasingly sophisticated. During driving, the vehicle's automatic headlights can turn on before the ambient light dims and turn off when the ambient light brightens. This helps reduce the driver's workload and minimizes potential safety hazards. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims. In a first aspect, this application provides an environmental visibility assessment method, the method comprising: acquiring a real-time image captured by a vehicle-mounted camera, and the actual shooting distance of the camera; identifying characters in the real-time image, extracting a first stroke feature of the character in the real-time image and a second stroke feature of the character in its complete form; determining the degree of stroke continuity of the first stroke feature relative to the second stroke feature, using the second stroke feature as a reference; and determining the visibility of the road environment based on the degree of stroke continuity and the actual shooting distance.

[0004] In one embodiment, identifying characters in the live image and extracting first stroke features of the characters in the live image and second stroke features of the characters in their complete form includes: performing semantic recognition processing on the live image to identify the characters; wherein the semantic recognition processing includes OCR recognition and / or skeleton recognition; and extracting the first stroke features and second stroke features based on the characters.

[0005] In one embodiment, performing OCR recognition on the live image includes: identifying a first region in the live image that is continuous and has no color change; performing shape recognition on a first region or a second region obtained by combining multiple first regions; and determining the character based on the recognized shape.

[0006] In one embodiment, skeletal recognition of the live image includes: identifying multiple pixel blocks in the live image that undergo color transitions; for each pixel block, determining the median coordinates of the pixel block based on the maximum and minimum values ​​of the pixel coordinates on the X and Y axes respectively, and identifying the pixels at the median coordinates as inflection points; sequentially connecting every two nearest inflection points to obtain skeletal information; and determining the character based on the skeletal information.

[0007] In one embodiment, the skeletal information is obtained by sequentially connecting every two nearest inflection points; determining the character based on the skeletal information includes: determining that every two nearest inflection points form an inflection point pair; performing vector operations on each inflection point pair to form a first stroke order matrix; comparing the first stroke order matrix with a preset second stroke order matrix, and determining the character based on the similarity between the two; wherein the preset second stroke order matrix is ​​associated with the corresponding character.

[0008] In one embodiment, after comparing the first stroke order matrix with a preset second stroke order matrix and determining the character based on their similarity, the method further includes: adding the distances between two inflection points in each inflection point pair corresponding to the first stroke order matrix to obtain a first stroke feature of the character; and adding the distances between two inflection points in each inflection point pair corresponding to the second stroke order matrix to obtain a second stroke feature.

[0009] In one embodiment, semantic recognition processing is performed on the live image to identify the character, including: performing OCR recognition and skeleton recognition on the live image to obtain OCR recognition results and skeleton recognition results respectively; determining whether the OCR recognition results and the skeleton recognition results are consistent; if the OCR recognition results and the skeleton recognition results are consistent, the semantic recognition processing is determined to be valid, and the information of the character is output; if the OCR recognition results and the skeleton recognition results are inconsistent, the semantic recognition processing is determined to be invalid, and the semantic recognition processing is re-performed based on the next frame of the live image.

[0010] In one embodiment, before performing semantic recognition processing on the live image, the method further includes: performing grayscale processing on the live image according to the R, G, and B channels to obtain a first grayscale image of the live image in the R, G, and B channels; performing convolution processing on the first grayscale image to obtain a second grayscale image; determining a target region in the second grayscale image whose pixel value is higher than a preset threshold, and cropping the live image to retain the target region; wherein the cropped live image is used as the input image for the semantic recognition processing.

[0011] In one embodiment, determining the visibility of the road environment at the time of acquisition of the live image based on the stroke continuity and the actual shooting distance includes: acquiring a preset dataset, wherein the preset dataset includes multiple reference data, each reference data including a shooting distance, stroke continuity, and environmental visibility that are related; comparing the information composed of the stroke continuity and the actual shooting distance with the preset dataset to obtain a comparison result, and determining the visibility of the road environment at the time of acquisition of the live image based on the comparison result.

[0012] In one embodiment, obtaining the preset dataset includes: setting preset scenes obtained by combining multiple different distance gradients and visibility gradients; acquiring simulated real-world images in each preset scene and extracting stroke features of characters in the simulated real-world images; wherein the stroke features include a third stroke feature and a fourth stroke feature, the third stroke feature corresponding to the preset scene with the highest visibility gradient; using the third stroke feature as a reference, determining the stroke continuity of the fourth stroke feature relative to the third stroke feature in each preset scene; and constructing the preset dataset based on the shooting distance, environmental visibility, and the calculated stroke continuity of the fourth stroke feature relative to the third stroke feature in each scene.

[0013] Secondly, this application provides a vehicle headlight control method, comprising: determining the visibility of the current road environment according to the environmental visibility assessment method described in the first aspect above; and controlling the operating state of the vehicle headlights according to the visibility of the current road environment.

[0014] Thirdly, this application provides an in-vehicle electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in the first aspect above.

[0015] The aforementioned environmental visibility assessment method, vehicle headlight control method, and onboard electronic device acquire real-time images of the road environment, identify characters in the images, extract actual first-stroke features and theoretical second-stroke features, determine the stroke continuity of the first-stroke features relative to the second-stroke features, and infer road environment visibility based on the stroke continuity of the first-stroke features and the actual shooting distance. By simulating human vision to assess environmental visibility, the problem of light source recognition errors can be avoided, thus improving the accuracy of environmental visibility assessment and better aligning with human eye habits. Other aspects will become clear after reading and understanding the accompanying drawings and detailed description. Attached Figure Description

[0016] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0017] Figure 1 is a hardware structure block diagram of an in-vehicle electronic device in one embodiment.

[0018] Figure 2 is a schematic diagram of the vehicle infotainment system in one embodiment.

[0019] Figure 3 is a flowchart of an environmental visibility assessment method in one embodiment.

[0020] Figure 4 is a schematic diagram of characters in one embodiment.

[0021] Figure 5 is a flowchart of an OCR recognition method in one embodiment.

[0022] Figure 6 is a schematic diagram of a real-world image from one embodiment.

[0023] Figure 7 is a flowchart of a skeletal recognition method in one embodiment.

[0024] Figure 8 is a schematic diagram of Chinese character recognition in one embodiment.

[0025] Figure 9 is a flowchart of an embodiment combining OCR recognition and skeletal recognition.

[0026] Figure 10 is a flowchart of a live image preprocessing process in one embodiment.

[0027] Figure 11 is a schematic diagram of a cropped real-world image in one embodiment. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0030] Current vehicle headlight control primarily relies on sensing ambient light intensity to assess road visibility and determine whether to activate the lights. Ambient light intensity is mainly sensed by photosensitive elements, which then send electronic signals to the controller to decide whether to turn on or off the vehicle's fog lights, low beam headlights, and / or high beam headlights. Because photosensitive elements only acquire light intensity information and have limited ability to identify light sources, they are easily affected by ambient light interference (such as strong reflections and alternating shadows), making headlight control errors prone to occur. Therefore, a more accurate method for assessing ambient visibility is needed to guide vehicle headlight operation.

[0031] In one embodiment, an in-vehicle electronic device is provided. This in-vehicle electronic device may be a vehicle central controller (e.g., a system-on-chip, or SOC), an in-vehicle electronic terminal, or a similar computing device. The method embodiments provided in this application can be executed in the in-vehicle electronic device. Figure 1 is a hardware structure block diagram of an in-vehicle electronic device according to an embodiment of this application. As shown in Figure 1, the in-vehicle electronic device may include one or more (only one is shown in Figure 1) processors 101 and a memory 102 for storing data. The processor 101 may include, but is not limited to, a microprocessor (MCU) or a field-programmable gate array (FPGA). The in-vehicle electronic device may also include a transmission device 103 for communication functions and an input / output device 104. Those skilled in the art will understand that the structure shown in Figure 1 is merely illustrative and does not limit the structure of the in-vehicle electronic device. For example, the in-vehicle electronic device may include more or fewer components than shown in Figure 1, or have a different configuration than shown in Figure 1.

[0032] The memory 102 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the environmental visibility assessment method in this embodiment. The processor 101 executes various functional applications and data processing by running the computer programs stored in the memory 102, thereby implementing the aforementioned method. The memory 102 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 102 may further include memory remotely located relative to the processor 101, and these remote memories can be connected to the vehicle's electronic devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0033] The transmission device 103 is used to receive or send data via a network. This network includes a wireless network provided by the communication provider of the vehicle electronics. In one example, the transmission device 103 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 103 can be a Radio Frequency (RF) module for wireless communication with the Internet.

[0034] In one embodiment, a vehicle infotainment system is provided. Figure 2 is a schematic diagram of the vehicle infotainment system. As shown in Figure 2, the vehicle infotainment system includes: an on-board electronic device 100, a sensor 200, an actuator controller 300, and a vehicle lighting system 400. The sensor 200 and the actuator controller 300 are respectively connected to the on-board electronic device 100, and the actuator controller 300 is also connected to the vehicle lighting system 400. The sensor 200 includes, but is not limited to, a light sensor, a camera (monocular and / or binocular), and radar (Lidar and / or Radar); the actuator controller 300 includes, but is not limited to, a motor and an optical mechanism; the vehicle lighting system 400 includes, but is not limited to, fog lights, high beams, low beams, and hazard lights. The vehicle electronic device 100 receives sensor data (such as real-time images and actual shooting distance) collected by the sensor 200, performs an environmental visibility assessment method to determine the visibility of the road environment, and sends a command to the execution controller 300 based on the visibility of the road environment, so that the execution controller 300 triggers a certain operating state of the vehicle lighting system 400, such as automatically switching between high and low beams or intelligently adjusting the lighting range and angle, in order to improve driving visibility.

[0035] In one embodiment, a vehicle headlight control method is provided, which determines the visibility of the current road environment according to the environmental visibility assessment method of this application; and controls the operating state of the vehicle headlights according to the visibility of the current road environment.

[0036] In practical applications, controlling the operation of vehicle lights based on the visibility of the current road environment includes automatically switching between high and low beams or intelligently adjusting the illumination range and angle. Automatic switching between high and low beams can occur when the assessed visibility is below a certain threshold, automatically switching between high and low beams to avoid interfering with other road users. Intelligent adjustment of the illumination range and angle can occur when the assessed visibility is below a certain threshold, automatically adjusting the illumination angle and range of the vehicle lights to provide optimal lighting effects. In this embodiment, by accurately assessing the visibility of the road environment and automatically controlling the operation of the vehicle lights based on the assessment results, the driver's field of vision can be improved, enhancing driving safety.

[0037] Figure 3 is a flowchart of the environmental visibility assessment method of this embodiment. Taking the application of this method to the above-mentioned vehicle electronic device as an example, it includes steps S101 to S104.

[0038] Step S101: Obtain the real-time image captured by the camera and the actual shooting distance of the camera.

[0039] The live image may contain images of target objects such as vehicles, road signs, speed limit signs, etc. The actual shooting distance of the camera is the distance between the in-vehicle camera and the target object, which can also be understood as the distance between the vehicle and the target object. The actual shooting distance can be obtained through the following methods: In the actual environment, after the vehicle recognizes the road sign information from the pictures collected by the camera, the binocular vision imaging module or sensors such as Lidar and Radar are used to calculate the position of the information carrier (i.e., the target object). For example, the high-speed navigation information on the road sign is recognized through the binocular vision imaging module, and based on the visual recognition result, the distance between the road sign and the vehicle is calculated using the binocular vision left and right parallax positions. Or, directly measure the distance to the nearest spatially highly reflective target (road sign or speed limit sign) through Lidar.

[0040] Step S102, recognize the characters in the live image, and extract the first stroke feature of the character in the live image and the second stroke feature of the character in the complete form.

[0041] It is possible to recognize the characters in the target object in the live image, and the recognized characters include but are not limited to classical Chinese (which can be languages and scripts of various countries), Arabic numerals, and symbols.

[0042] Both the first stroke feature and the second stroke feature are the total stroke lengths of the character. Among them, the first stroke feature is the total stroke length actually calculated for the character in the live image, and the second stroke feature is the total stroke length theoretically calculated for the character in the complete size situation (complete form). When the environmental visibility is low (weak ambient light, turbid air or far distance), the characters recognized in the visual imaging will be缺损, resulting in discontinuous stroke order. For easy understanding, Figure 4 provides a schematic diagram of a character. As shown in Figure 4, assume that the character "田" is recognized based on the live image. The left part represents the缺损 shape of the character, and the right part represents the complete shape of the character. Then the first stroke feature L1 is the total length of the discontinuous stroke order in the actual situation, and the second stroke feature L2 is the total length of the continuous stroke order in the假想 situation, that is, the second stroke feature L2 is the total length of the continuous stroke order of the character in the complete form.

[0043] In this step, semantic recognition processing can be performed on the live image to recognize the characters; among them, the semantic recognition processing includes OCR recognition and / or skeleton recognition; the first stroke feature and the second stroke feature are extracted according to the characters. Specifically, a convolutional neural network (CNN) can be used to perform target recognition and differentiation on the collected live image.

[0044] Step S103, taking the second stroke feature as the reference object, determine the degree of stroke continuity of the first stroke feature relative to the second stroke feature.

[0045] It should be noted that the text contains some unclear or incorrect expressions in Chinese (such as "缺损" and "假想"), which may affect the accuracy of the overall translation. It is recommended to clarify and correct these parts in the original text for a more accurate translation.For each recognized character, the actual calculated total stroke length L1 is divided by the theoretically calculated total stroke length L2 to obtain the stroke continuity of the character. The more severe the character's defects, the lower its stroke continuity; conversely, the less severe the character's defects, the higher its stroke continuity.

[0046] Step S104: Determine the visibility of the road environment at the time of image acquisition based on the continuity of strokes and the actual shooting distance.

[0047] Stroke continuity is positively correlated with environmental visibility; higher environmental visibility results in higher stroke continuity. Stroke continuity is also related to shooting distance; closer shooting distances result in higher stroke continuity. Therefore, a preset dataset can be obtained, comprising multiple reference data points, each including correlated shooting distance, stroke continuity, and environmental visibility. The information derived from stroke continuity and actual shooting distance is compared with the preset dataset to obtain the comparison result, which determines the visibility of the current road environment. It can be understood that a higher stroke continuity for the first stroke feature and a closer shooting distance will result in higher road environment visibility.

[0048] In steps S101 to S104 above, by acquiring real-time images of the road environment, identifying characters in the real-time images, and extracting the actual first stroke features and theoretical second stroke features, the degree of stroke continuity of the first stroke features relative to the second stroke features is determined. Based on the degree of stroke continuity and the actual shooting distance, the visibility of the road environment is inferred. By simulating human vision to evaluate environmental visibility, the problem of light source recognition errors can be avoided, thus improving the accuracy of environmental visibility assessment and better aligning with human eye habits.

[0049] In some embodiments, characters may be recognized using only Optical Character Recognition (OCR), only using skeletal character recognition, or both OCR and skeletal character recognition methods. These different scenarios will be explained below.

[0050] (1) OCR recognition method:

[0051] In one embodiment, Figure 5 provides a flowchart of an OCR recognition method. As shown in Figure 5, OCR recognition of a real-world image includes steps S201 to S202.

[0052] Step S201: In the live image, determine a first region where the color remains unchanged and is continuous;

[0053] Step S202, perform shape recognition on the first region or the second region obtained by combining multiple first regions, and determine the characters in the live image according to the recognized shape.

[0054] For ease of understanding, FIG. 6 provides a schematic diagram of a live image. As shown in FIG. 6, taking a road sign as an example, assuming that the character "庆" exists in the live image, it can be found that the internal colors of the radicals "广" and "大" of this character are white and continuous, while the colors between "广" and "大" are other colors adjacent but discontinuous. Then, "广" and "大" can be regarded as the first regions respectively, and then these two first regions are combined to obtain the second region. By performing shape recognition on the second region, the corresponding character can be recognized. Of course, some characters only have the first region and do not need to be combined to obtain the second region. For example, "上" and "了", then only shape recognition needs to be performed on the first region.

[0055] Taking the image recognition of digital information and letter information as an example, based on the coordinates of the color convolution result, first perform image region division. Regions with unchanged and continuous colors are divided into one region, and regions with unchanged but discontinuous colors are divided into one region (i.e., the second region). Subsequently, optical character recognition is performed using the image shape to complete the recognition of semantic texts such as road signs and speed limit signs.

[0056] (2) Skeleton recognition method:

[0057] In one embodiment, FIG. 7 provides a flowchart of a skeleton recognition method. As shown in FIG. 7, performing skeleton recognition on the live image includes steps S301 to S303.

[0058] Step S301, identify multiple pixel blocks with color jumps in the live image.

[0059] FIG. 8 provides a schematic diagram of character recognition. As shown in FIG. 8, taking the example that the character "田" is included in the live image, the region of interest in the live image ("田") can be determined by the region of interest recognition method, that is, the image determined by the pixels [x1, y1] to [x2, y2]. Then, in the RGB channel map of the live image, for each x coordinate from x1 to x2 and each y coordinate from y1 to y2, a pixel block with each color jump is recognized using an operator with a specific size. Among them, the operator with a specific size refers to a matrix with a specific number of rows and columns, and the matrix size is determined according to the image size after image cropping processing. For example, the cropped live image can be evenly divided along the X-axis and Y-axis directions to obtain the number of equal divisions, which can be set by the vehicle-mounted electronic terminal. The operator can also be understood as a sliding window for checking the relationship between pixels and their surrounding pixels.

[0060] Step S302: For each of the multiple pixel blocks, determine the median coordinates corresponding to the pixel block according to the maximum and minimum values of the pixel coordinates of the pixel block on the X-axis and Y-axis respectively, and determine the pixel at the median coordinates as the inflection point.

[0061] An inflection point can refer to the starting and ending points of a certain area in an image. Or, starting from a certain skeletal point in the image, extending in four directions: the positive and negative directions of the X-axis and the positive and negative directions of the Y-axis of the image. If there are two other skeletal points whose connection vector directions with this skeletal point are different, then this skeletal point is determined as an inflection point. For example, if the connection direction between the skeletal point (x1, y1) and the right-side skeletal point is horizontally to the right (0°), and the connection direction between the skeletal point (x1, y1) and the lower skeletal point is vertically downward (90°), then the direction difference is obvious, and the skeletal point (x1, y1) is an inflection point. Among them, a skeletal point refers to the key point obtained after skeletonizing the object of interest in the image. The key point is located on the center line of the object of interest and is used to reflect the basic shape and structural characteristics of the object of interest. For a character, an inflection point is the point of change, starting point, and ending point of its stroke direction; for a number, an inflection point is the boundary point, starting point, and ending point of its stroke.

[0062] Specifically, referring to FIG. 8, obtain the maximum and minimum values of the coordinates of each pixel block with color jump in the X-axis and Y-axis directions, perform median processing on the maximum and minimum values of the coordinates to obtain the inflection point. Taking the character "田" as an example, the calculation is as follows: The midpoint of (x1, y1) and (x3, y3) is A[(x1 + x3) / 2, (y1 + y3) / 2]; the midpoint of (x2, y1) and (x6, y3) is B[(x2 + x6) / 2, (y1 + y3) / 2]; the midpoint of (x1, y2) and (x3, y6) is C[(x1 + x3) / 2, (y2 + y6) / 2]; the midpoint of (x6, y6) and (x2, y2) is D[(x2 + x6) / 2, (y6 + y2) / 2]; the midpoint of (x3, y4) and (x3, y5) is E[(x3 + x3) / 2, (y4 + y5) / 2]; the midpoint of (x4, y3) and (x5, y3) is F[(x4 + x5) / 2, (y3 + y3) / 2]; the midpoint of (x4, y6) and (x5, y6) is G[(x4 + x5) / 2, (y6 + y6) / 2]; the midpoint of (x6, y4) and (x6, y5) is H[(x6 + x6) / 2, (y4 + y5) / 2].

[0063] Among them, the midpoints of the line segments formed by E and H, F and G are both O, A, B, D, C are image boundary points (also belonging to inflection points), and E, F, G, H, O are all inflection points.

[0064] Step S303: Connect every two nearest inflection points in sequence to obtain skeletal information; determine the characters in the live image based on the skeletal information.

[0065] Each pair of closest inflection points is identified as an inflection point pair. For each inflection point pair, vector operations are performed to form a first stroke order matrix. The first stroke order matrix is ​​compared with a pre-defined second stroke order matrix, and the corresponding character is determined based on their similarity. The second stroke order matrix is ​​associated with the corresponding character. After comparing the first stroke order matrix with the pre-defined second stroke order matrix and determining the corresponding character based on their similarity, the distances between the two inflection points in each inflection point pair corresponding to the first stroke order matrix are added to obtain the first stroke feature of the character; the distances between the two inflection points in each inflection point pair corresponding to the second stroke order matrix are added to obtain the second stroke feature.

[0066] In practical applications, for a live image that has already undergone region of interest (ROI) identification and cropping, starting from the top left corner of the live image, for each inflection point, find the nearest other inflection point, and perform vector operations on each pair of inflection points to obtain the entire skeletal path, which is the first stroke order matrix. Based on the same principle as the above method, perform skeletal recognition on a clear live image, or perform skeletal recognition on an image from an existing character library, to obtain a second stroke order matrix. Pre-store multiple second stroke order matrices to obtain a standard stroke order matrix library. Iterate and compare the first stroke order matrix with the second stroke order matrices in the standard stroke order matrix library, and take the character associated with the second stroke order matrix with the highest similarity as the recognized character.

[0067] (3) Combining OCR recognition with skeletal recognition:

[0068] In one embodiment, Figure 9 provides a flowchart of an OCR recognition method combined with a skeleton recognition method, as shown in Figure 9, including steps S401 to S405.

[0069] Step S401: Perform OCR recognition on the live image to obtain the OCR recognition result; see Figure 5 for details.

[0070] Step S402: Perform skeletal recognition on the real-world image to obtain the skeletal recognition result; see Figure 7 for details.

[0071] Step S403: Determine whether the OCR recognition result and the skeleton recognition result are consistent; if they are consistent, proceed to step S404; if they are inconsistent, proceed to step S405.

[0072] Step S404: Determine that the semantic recognition process is effective and output the corresponding character information.

[0073] Step S405: Determine that the semantic recognition process is invalid, and re-perform the semantic recognition process based on the next frame of the live image.

[0074] In this embodiment, characters identified using skeletal recognition are compared with those identified using OCR. If the results of the two algorithms are identical, the recognition is deemed valid, and character information is output. Character information can be the character itself (literal Chinese, Arabic numerals, symbols) or the boundary points of the character, where boundary points refer to the coordinates of the stroke boundary points of a complete character. If the results of the two algorithms differ, semantic recognition will use the next frame of the live image for re-recognition until the correct character information is output, or the target object in the road environment moves out of the camera's field of view. This setup considers that skeletal recognition performs better in handling complex backgrounds and partial occlusion, while OCR is more accurate when handling clear, standard characters. When the recognition results of the two algorithms are consistent, the recognition result can be considered reliable, and character information is output. This dual verification mechanism significantly reduces the probability of misrecognition. If the results of the two algorithms are inconsistent, the next frame of the image will be used for re-recognition until the results of the two methods are consistent or the target object moves out of the field of view. This dynamic adjustment and continuous verification process further improves the accuracy of recognition. In practical applications, characters in road environments may be affected by various factors, such as changes in lighting, occlusion, and dirt. Multi-algorithm fusion methods can better handle these complex situations and improve the system's robustness. This embodiment, through multi-algorithm fusion, complementarity, dual verification, robustness, and real-time design, significantly improves the accuracy of semantic recognition, ensuring reliable character information recognition in various complex environments.

[0075] In one embodiment, FIG10 provides a flowchart of preprocessing a live image. As shown in FIG10, before performing semantic recognition processing on the live image, the method further includes steps S501 to S503.

[0076] Step S501: Perform grayscale processing on the live image according to the three channels R, G, and B to obtain the first grayscale image of the live image on the three channels R, G, and B.

[0077] The acquired live image can be grayscaled in three color layers: R (red), G (green), and B (blue) to obtain the grayscale distribution of the live image in the three color layers. For white areas with green as the background in the live image, they will be displayed as highlights after grayscale conversion in the three RGB image layers.

[0078] Referring to Figure 6, taking road signs as the target object as an example, in actual road testing, a large number of real-world images containing the target object can be collected as sample images for model training, and the color information of the target object in the sample images can be extracted. For example, when green represents the background, information such as text, numbers, destination distance, and direction is white; or, when blue represents the background, information such as text, numbers, destination distance, and direction is white.

[0079] Step S502: Perform convolution processing on the first grayscale image to obtain the second grayscale image.

[0080] For example, on the green layer, a convolution operation is performed using a graphics window of a certain size. Because green has a high grayscale value in layer G, the high grayscale area in the second grayscale image after convolution represents the green road sign, thus obtaining the pixel coordinates [x1, y1] to [x2, y2] of the road sign in the real-world image. Since the white text and numbers in the green area have consistent grayscale values ​​in the RGB layers (white has consistent positional characteristics in grayscale division), similar coordinate positions can be obtained in the three color layers. Similarly, for target objects of other colors, the same color determination method can be used to obtain the coordinate position of the target object in the image.

[0081] Step S503: Determine the target region in the second grayscale image whose pixel value is higher than a preset threshold, and crop the live image to retain the target region; wherein, the cropped live image is used as the input image for semantic recognition processing. In some embodiments, by setting a grayscale value threshold (e.g., 128), all regions in the second grayscale image whose pixel values ​​are greater than the threshold are marked as target regions, that is, the pixel values ​​in the target regions are all higher than the preset threshold.

[0082] Figure 11 is a schematic diagram of the cropped live image. As shown in Figure 11, based on the coordinate information of the target object already obtained by the convolutional layer, the original image size is preserved according to [x1, y1] to [x2, y2] to form a new image, which is then passed to the downstream (semantic recognition) for further processing. In practical applications, an original live image usually contains multiple characters, so multiple sub-images are cropped for each character. The environmental visibility can be evaluated by combining the stroke feature continuity of multiple sub-images. Specifically, the minimum environmental visibility evaluation value can be taken as the current environmental visibility, or the average of the multiple environmental visibility evaluation values ​​can be taken as the current environmental visibility. This embodiment does not impose any restrictions.

[0083] This embodiment separates the original image by recognizing the color features of the target object, reducing the image size while retaining the information of the target object, which helps to reduce the computational load of subsequent semantic recognition processing.

[0084] In one embodiment, obtaining a preset dataset includes: setting preset scenes obtained by combining multiple different distance gradients and visibility gradients; acquiring simulated real-world images in each preset scene and extracting stroke features of characters in the simulated real-world images; wherein the stroke features include a third stroke feature and a fourth stroke feature, the third stroke feature corresponding to the preset scene with the highest visibility gradient; using the third stroke feature as a reference, determining the stroke continuity of the fourth stroke feature relative to the third stroke feature in each preset scene; and constructing a preset dataset based on the shooting distance, environmental visibility, and the calculated stroke continuity of the fourth stroke feature relative to the third stroke feature in each preset scene.

[0085] In this embodiment, the stroke continuity of the vehicle under different distance and visibility gradients can be pre-calibrated. For example, nine different test scenarios can be constructed in a test field based on two dimensions: distance (far, medium, near) and visibility (low, medium, high) to calibrate the vehicle's ability to recognize target objects. In each scenario, the vehicle's camera captures images of the target object, and target recognition is performed on the images under different visibility conditions, particularly recognizing characters on the target object. For high visibility scenarios, the total stroke length L1 of the recognized characters is calculated; similarly, the above process is repeated under medium and low visibility conditions, calculating the total stroke lengths L2 and L3, respectively. By comparing the ratios of L2 / L1 and L3 / L1, the stroke continuity Q under different visibility conditions can be evaluated. Thus, a matrix [distance, stroke continuity, environmental visibility] containing distance, stroke continuity, and environmental visibility can be generated to analyze the vehicle's recognition performance of target objects under different conditions.

[0086] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon. When executed by a processor, the computer program performs the following steps: acquiring a live image captured by a vehicle-mounted camera and the actual shooting distance of the camera; identifying characters in the live image and extracting a first stroke feature of the character in the live image and a second stroke feature of the character in its complete form; determining the degree of stroke continuity of the first stroke feature relative to the second stroke feature using the second stroke feature as a reference; and determining the visibility of the road environment based on the degree of stroke continuity and the actual shooting distance.

[0087] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0088] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0089] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for assessing environmental visibility, comprising: Acquire real-time images captured by the vehicle-mounted camera, as well as the actual shooting distance of the camera; Identify characters in the live image, and extract the first stroke features of the characters in the live image and the second stroke features of the characters in their complete form; Using the second stroke feature as a reference, determine the degree of stroke continuity of the first stroke feature relative to the second stroke feature; Based on the continuity of the strokes and the actual shooting distance, the visibility of the road environment at the time of acquisition of the live image is determined.

2. The environmental visibility assessment method according to claim 1, wherein, Identify characters in the live image, and extract the first stroke features of the characters in the live image and the second stroke features of the characters in their complete form, including: The live image is subjected to semantic recognition processing to identify the characters; wherein, the semantic recognition processing includes OCR recognition and / or skeleton recognition; Extract the first stroke feature and the second stroke feature based on the character.

3. The environmental visibility assessment method according to claim 2, wherein, Performing OCR recognition on the live image includes: In the live image, a first region whose color remains unchanged and is continuous is identified; Shape recognition is performed on one of the first regions or a second region obtained by combining multiple first regions, and the character is determined based on the recognized shape.

4. The environmental visibility assessment method according to claim 2, wherein, Performing skeletal recognition on the live image includes: Multiple pixel blocks that exhibited color jumps were identified in the live image; For each of the plurality of pixel blocks, Based on the maximum and minimum values ​​of the pixel coordinates on the X and Y axes of the pixel block, determine the median coordinates of the corresponding pixel block, and The pixel located at the median coordinates is determined as the inflection point; By sequentially connecting every two nearest inflection points, skeletal information is obtained; The character is determined based on the skeletal information.

5. The environmental visibility assessment method according to claim 4, wherein, By sequentially connecting every two nearest inflection points, the skeletal information is obtained; Determining the character based on the skeletal information includes: Determine that every two closest inflection points form an inflection point pair. For each inflection point pair, perform vector operations on the inflection point pair to form the first stroke order matrix. The first stroke order matrix is ​​compared with a preset second stroke order matrix, and the character is determined based on the similarity between the two; wherein the preset second stroke order matrix is ​​associated with the corresponding character.

6. The environmental visibility assessment method according to claim 5, wherein, After comparing the first stroke order matrix with a preset second stroke order matrix and determining the character based on their similarity, the method further includes: The distances between the two inflection points in each inflection point pair corresponding to the first stroke order matrix are added together to obtain the first stroke feature of the character; The distances between two inflection points in each inflection point pair corresponding to the preset second stroke order matrix are added together to obtain the second stroke feature.

7. The environmental visibility assessment method according to claim 2, wherein, The live image is subjected to semantic recognition processing to identify the characters, including: The live image is subjected to OCR recognition and skeleton recognition to obtain OCR recognition results and skeleton recognition results respectively; Determine whether the OCR recognition result and the skeleton recognition result are consistent; If the OCR recognition result and the skeleton recognition result are consistent, then the semantic recognition processing is determined to be effective, and the information of the character is output. If the OCR recognition result and the skeleton recognition result are inconsistent, the semantic recognition process is determined to be invalid, and the semantic recognition process is re-performed based on the next frame of the live image.

8. The environmental visibility assessment method according to claim 2, wherein, Before performing semantic recognition processing on the live image, the method further includes: The live image is processed into grayscale according to the three channels R, G, and B to obtain the first grayscale image of the live image in the three channels R, G, and B. The first grayscale image is convolved to obtain the second grayscale image. A target region with a pixel value higher than a preset threshold is identified in the second grayscale image, and the live image is cropped to retain the target region; wherein the cropped live image is used as the input image for the semantic recognition processing.

9. The environmental visibility assessment method according to any one of claims 1 to 8, wherein, Based on the continuity of the strokes and the actual shooting distance, the visibility of the road environment at the time of acquisition of the live image is determined, including: Obtain a preset dataset, wherein the preset dataset includes multiple reference data, each of which includes related shooting distance, stroke continuity and environmental visibility; The information consisting of the continuity of the strokes and the actual shooting distance is compared with the preset dataset to obtain the comparison result. Based on the comparison result, the visibility of the road environment at the time of acquisition of the real-time image is determined.

10. The environmental visibility assessment method according to claim 9, wherein, Obtaining the preset dataset includes: Set up a preset scene obtained by combining various distance gradients and visibility gradients; Simulated real-world images are acquired in each of the preset scenarios, and stroke features of characters in the simulated real-world images are extracted; wherein, the stroke features include a third stroke feature and a fourth stroke feature, and the third stroke feature corresponds to the preset scenario with the highest visibility gradient; Using the third stroke feature as a reference, determine the degree of stroke continuity of the fourth stroke feature relative to the third stroke feature in each preset scenario; The preset dataset is constructed based on the shooting distance, environmental visibility, and the calculated stroke continuity of the fourth stroke feature relative to the third stroke feature for each preset scene.

11. A method for controlling vehicle lights, comprising: The environmental visibility assessment method according to any one of claims 1 to 10 determines the visibility of the current road environment; The vehicle lights are controlled according to the visibility of the current road environment.

12. An in-vehicle electronic device, comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method according to any one of claims 1 to 11.

Images