Environmental visibility evaluation method, vehicle lamp control method and vehicle-mounted electronic device

Abstract

The invention relates to an environment visibility evaluation method, a vehicle lamp control method and a vehicle-mounted electronic device. The method comprises the following steps: acquiring a live image obtained by acquiring a road environment by a camera and an actual shooting distance of the camera; identifying a character in the live image, extracting a first stroke feature of the character, and extracting a second stroke feature of the character under the condition of complete size; determining the stroke continuity degree of the first stroke feature relative to the second stroke feature by taking the second stroke feature as a reference object; determining the visibility of the road environment according to the stroke continuity degree of the first stroke feature and the actual shooting distance; and the accuracy of environmental visibility evaluation is improved.

Description

Technical Field

[0001] The present application relates to the technical field of vehicle safe driving, and in particular to an environmental visibility assessment method, a vehicle light control method and a vehicle-mounted electronic device. Background Art

[0002] With the development of automotive electronic technology, the safety protection measures of automobiles are becoming more and more perfect. During the driving process, the vehicle's automatic sensing headlights can automatically light up before the ambient light becomes dark, and automatically turn off when the ambient light becomes brighter. This helps to reduce the driver's vehicle control intensity and reduce driving safety hazards.

[0003] At present, headlight control mainly evaluates road environment visibility by sensing ambient light intensity, thereby deciding whether to turn on the headlights. The ambient light intensity is mainly sensed by photosensitive elements, which send electronic signals to the controller to decide whether to turn on or off the vehicle headlights. Since photosensitive elements only obtain light intensity information and have low recognition of light sources, it is easy to cause errors in headlight control. Therefore, a more accurate environmental visibility assessment method is needed to guide the operating status of headlights.

[0004] At present, no effective solution has been proposed to address the problem that the road environment visibility assessment methods of related technologies are not accurate enough. Summary of the invention

[0005] Based on this, it is necessary to provide an environmental visibility assessment method, a vehicle light control method and a vehicle-mounted electronic device that can accurately assess the road environment visibility in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for evaluating environmental visibility, the method comprising:

[0007] Acquire a real-time image of the road environment captured by a camera, and an actual shooting distance of the camera;

[0008] Recognize characters in the live image, extract first stroke features of the characters, and extract second stroke features of the characters in full size;

[0009] Taking the second stroke feature as a reference object, determining the stroke continuity degree of the first stroke feature relative to the second stroke feature;

[0010] The visibility of the road environment is determined according to the stroke continuity of the first stroke feature and the actual shooting distance.

[0011] In one embodiment, recognizing characters in the live image, extracting first stroke features of the characters, and extracting second stroke features of the characters in full size include:

[0012] Performing semantic recognition processing on the live image to identify the characters; wherein the semantic recognition processing includes OCR recognition and / or skeleton recognition;

[0013] The first stroke feature and the second stroke feature are obtained according to the character extraction.

[0014] In one embodiment, performing OCR recognition on the live image includes:

[0015] In the live image, determining a first region whose color does not change and is continuous;

[0016] Shape recognition is performed on the first area or the second area obtained by combining the first areas, and a corresponding character is determined according to the recognized shape.

[0017] In one embodiment, performing skeleton recognition on the live image includes:

[0018] Identifying pixel blocks where color jump occurs in the live image;

[0019] Determine median coordinates according to the maximum and minimum values ​​of pixel coordinates in the pixel block on the X-axis and the Y-axis, respectively, and determine the pixel at the median coordinates as an inflection point;

[0020] Connecting every two inflection points that are closest to each other in sequence to obtain skeleton information;

[0021] The corresponding character is determined according to the skeleton information.

[0022] In one embodiment, every two inflection points that are closest to each other are sequentially connected to obtain skeleton information; and the corresponding character is determined according to the skeleton information, including:

[0023] Determine every two pairs of inflection points that are closest to each other, perform vector operations on each pair of inflection points, and form a first stroke order matrix;

[0024] The first stroke order matrix is ​​compared with a preset second stroke order matrix, and corresponding characters are determined according to the similarity between the two; wherein the second stroke order matrix is ​​associated with corresponding characters.

[0025] In one embodiment, after comparing the first stroke order matrix with a preset second stroke order matrix and determining corresponding characters according to the similarity between the two, the method further includes:

[0026] Adding the distances between two inflection points in each inflection point pair corresponding to the first stroke order matrix to obtain a first stroke feature of the character;

[0027] The distances between two inflection points in each inflection point pair corresponding to the second stroke order matrix are added to obtain the second stroke feature.

[0028] In one embodiment, performing semantic recognition processing on the live image to recognize the characters includes:

[0029] Performing OCR recognition and skeleton recognition on the live image to obtain an OCR recognition result and a skeleton recognition result respectively;

[0030] Determine whether the OCR recognition result is consistent with the skeleton recognition result;

[0031] If the OCR recognition result is consistent with the skeleton recognition result, it is determined that the current semantic recognition process is valid, and the corresponding character information is output;

[0032] If the OCR recognition result is inconsistent with the skeleton recognition result, it is determined that the current semantic recognition process is invalid, and the semantic recognition process is performed again based on the next frame of live image.

[0033] In one embodiment, before performing semantic recognition processing on the live image, the method further includes:

[0034] Performing grayscale processing on the live image according to the three channels of R, G, and B to obtain a first grayscale image of the live image on the three channels of R, G, and B;

[0035] Performing convolution processing on the first grayscale image to obtain a second grayscale image;

[0036] Determine a target area in the second grayscale image whose pixel values ​​are higher than a preset threshold, and crop the live image to retain the target area; wherein the cropped live image is used as an input image for the semantic recognition process.

[0037] In one embodiment, determining the visibility of the road environment according to the stroke continuity of the first stroke feature and the actual shooting distance includes:

[0038] Acquire a preset data set, wherein the preset data set includes a plurality of reference data, each of which includes mutually related shooting distances, stroke continuity, and environmental visibility;

[0039] The information formed by the stroke continuity of the first stroke feature and the actual shooting distance is compared with a preset data set, and the visibility of the current road environment is determined according to the comparison result.

[0040] In one embodiment, obtaining a preset data set includes:

[0041] Set up scenes with multiple combinations of distance gradients and visibility gradients;

[0042] Collecting live images in each of the scenes, and extracting stroke features of characters in the live images; wherein the stroke features include a third stroke feature and a fourth stroke feature, and the third stroke feature corresponds to a scene with the highest visibility gradient;

[0043] Taking the third stroke feature as a reference object, determining the stroke continuity degree of the fourth stroke feature relative to the third stroke feature in each scene;

[0044] The preset data set is constructed according to the shooting distance of each scene, the environmental visibility and the calculated stroke continuity of the fourth stroke feature.

[0045] In a second aspect, the present application provides a vehicle light control method, comprising:

[0046] Determine the visibility of the current road environment according to the environment visibility assessment method described in the first aspect above;

[0047] Control the operating status of the vehicle headlights according to the visibility of the current road environment.

[0048] In a third aspect, the present application provides a vehicle-mounted electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect are implemented.

[0049] The above-mentioned environmental visibility evaluation method, vehicle light control method and vehicle-mounted electronic device collect live images of the road environment, recognize characters in the live images and extract the actual first stroke features and the theoretical second stroke features, determine the stroke continuity of the first stroke features relative to the second stroke features, and reversely infer the visibility of the road environment based on the stroke continuity of the first stroke features and the actual shooting distance. By evaluating environmental visibility in a way that simulates human vision, the problem of light source recognition errors can be avoided, thereby improving the accuracy of environmental visibility evaluation and being more in line with human eye habits. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A hardware structure block diagram of a vehicle-mounted electronic device in one embodiment;

[0051] Figure 2 A schematic diagram of the structure of a vehicle system in an embodiment;

[0052] Figure 3 is a flow chart of an environmental visibility assessment method in one embodiment;

[0053] Figure 4is a character schematic diagram in one embodiment;

[0054] Figure 5 is a flow chart of an OCR recognition method in one embodiment;

[0055] Figure 6 is a schematic diagram of a live image in one embodiment;

[0056] Figure 7 is a flow chart of a skeleton recognition method in one embodiment;

[0057] Figure 8 A schematic diagram of character recognition in an embodiment;

[0058] Fig. 9 A flowchart of an OCR recognition method combined with a skeleton recognition method in one embodiment;

[0059] Fig.10 is a flowchart of live image preprocessing in one embodiment;

[0060] Fig.11 FIG. 1 is a schematic diagram of a cropped live image in an embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0063] In one embodiment, a vehicle-mounted electronic device is provided. The vehicle-mounted electronic device may be a vehicle central control unit (SOC), a vehicle-mounted electronic terminal or a similar computing device. The method embodiments provided in the present application may be executed in the vehicle-mounted electronic device. Figure 1 1 is a hardware structure block diagram of a vehicle-mounted electronic device according to an embodiment of the present application. Figure 1 As shown, the vehicle-mounted electronic device may include one or more ( Figure 1 Only one is shown in the figure) processor 101 and memory 102 for storing data, wherein processor 101 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above-mentioned vehicle-mounted electronic device may also include a transmission device 103 and an input and output device 104 for communication functions. It can be understood by ordinary technicians in this field that Figure 1 The structure shown is for illustration only and does not limit the structure of the above-mentioned vehicle-mounted electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0064] The memory 102 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the environmental visibility assessment method in the present embodiment. The processor 101 executes various functional applications and data processing by running the computer program stored in the memory 102, that is, to implement the above method. The memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 102 may further include a memory remotely arranged relative to the processor 101, and these remote memories may be connected to the vehicle-mounted electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0065] The transmission device 103 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the vehicle-mounted electronic device. In one embodiment, the transmission device 103 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one embodiment, the transmission device 103 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0066] In one embodiment, a vehicle system is provided. Figure 2 It is a structural diagram of the vehicle system. Figure 2 As shown, the vehicle system includes: an on-board electronic device 100, a sensor 200, an execution controller 300 and a vehicle light system 400; the sensor 200 and the execution controller 300 are respectively connected to the on-board electronic device 100, and the execution controller 300 is also connected to the vehicle light system 400. Among them, the sensor 200 includes but is not limited to a light sensor, a camera (monocular and / or binocular), and a radar (Lidar and / or Radar); the execution controller 300 includes but is not limited to a motor and an optical mechanism; the vehicle light system 400 includes but is not limited to a fog lamp, a high beam, a low beam, and a flash. The on-board electronic device 100 receives the sensor data (such as a live image, an actual shooting distance) collected by the sensor 200, executes an environmental visibility assessment method, determines the visibility of the road environment, and issues instructions to the execution controller 300 based on the visibility of the road environment, so that the execution controller 300 triggers a certain operating state of the vehicle light system 400, such as automatically switching between high and low beams or intelligently adjusting the lighting range and angle, etc., to improve driving visibility.

[0067] In one embodiment, a vehicle headlight control method is provided, which determines the visibility of the current road environment according to the environmental visibility assessment method of the present application; and controls the operating state of the vehicle headlights according to the visibility of the current road environment.

[0068] In actual applications, when the operating state of the vehicle headlights is controlled according to the visibility of the current road environment, it includes automatically switching between high and low beams or intelligently adjusting the lighting range and angle. Among them, the automatic switching between high and low beams can be when the assessed environmental visibility is lower than a certain threshold, the high and low beams are automatically switched to avoid interference with other road users. Intelligent adjustment of the lighting range and angle can be when the assessed environmental visibility is lower than a certain threshold, the lighting angle and lighting range of the headlights are automatically adjusted to provide the best lighting effect. In this embodiment, by accurately evaluating the visibility of the road environment and automatically controlling the operating state of the vehicle headlights based on the evaluation results, the driver's field of vision can be improved and driving safety can be enhanced.

[0069] Figure 3 Flowchart of the environment visibility assessment method of this embodiment is shown in FIG. 1 , which is described by taking the method applied to the above-mentioned vehicle-mounted electronic device as an example, and includes the following steps:

[0070] Step S101, obtaining a real-time image of the road environment captured by a camera and an actual shooting distance of the camera.

[0071] The live image may contain images of target objects such as vehicles, road signs, and speed limit signs. The actual shooting distance of the camera is the distance between the camera or the vehicle and the target object. The actual shooting distance can be obtained in the following ways:

[0072] In the actual environment, after the vehicle recognizes the road sign information through the pictures collected by the camera, it uses the binocular vision imaging module or Lidar, Radar and other sensors to measure the position of the information carrier (i.e. the target object). For example, the high-speed navigation information on the road sign is recognized by the binocular vision imaging module, and based on the visual recognition results, the distance between the road sign and the vehicle is calculated using the binocular vision left and right aberration position. Alternatively, the distance to the nearest high-reflectivity target (road sign or speed limit sign) in space is directly measured by Lidar.

[0073] Step S102, identifying characters in the live image, extracting first stroke features of the characters, and extracting second stroke features of the characters in full size.

[0074] The characters include, but are not limited to, classical Chinese (which can be in various languages), Arabic numerals, and symbols.

[0075] The first stroke feature and the second stroke feature are both the total stroke lengths of characters. Among them, the first stroke feature is the total stroke length actually calculated for the characters in the live image, and the second stroke feature is the total stroke length theoretically calculated for the characters in the complete size case. When the environmental visibility is low (weak ambient light, turbid air, or long distance), the characters recognized in visual imaging will be defective, resulting in discontinuous stroke order. For ease of understanding, Figure 4 a character schematic diagram is provided, such as Figure 4 shown. Assuming that the character "field" is recognized based on the live image, the left part represents the defective shape of the character, and the right part represents the complete shape of the character. Then the first stroke feature L1 is the total length of the discontinuous stroke order in the actual situation, and the second stroke feature L2 is the total length of the continuous stroke order in the imaginary situation.

[0076] In this step, semantic recognition processing can be performed on the live image to recognize the characters. Among them, the semantic recognition processing includes OCR recognition and / or skeleton recognition; the first stroke feature and the second stroke feature are extracted according to the characters. Specifically, a convolutional neural network (CNN) can be used to perform object recognition and differentiation on the collected live image.

[0077] Step S103: Taking the second stroke feature as the reference object, determine the stroke continuity degree of the first stroke feature relative to the second stroke feature.

[0078] For the recognized characters, divide the total stroke length L1 actually calculated by the total stroke length L2 theoretically calculated to obtain the stroke continuity degree of the characters. The more serious the character defect situation is, the lower its stroke continuity degree is. On the contrary, the milder the character defect situation is, the higher its stroke continuity degree is.

[0079] Step S104: Determine the visibility of the road environment according to the stroke continuity degree of the first stroke feature and the actual shooting distance.

[0080] The stroke continuity degree and the environmental visibility are positively correlated. The higher the environmental visibility is, the higher the stroke continuity degree is. The stroke continuity degree is also related to the shooting distance. The closer the shooting distance is, the higher the stroke continuity degree is. Therefore, by obtaining a preset data set, the preset data set includes multiple reference data, and each reference data includes the related shooting distance, stroke continuity degree, and environmental visibility; compare the information composed of the stroke continuity degree of the first stroke feature and the actual shooting distance with the preset data set, and determine the visibility of the current road environment according to the comparison result. It can be understood that when the stroke continuity degree of the first stroke feature is higher and the shooting distance is closer, the inferred visibility of the road environment will be higher.

[0081] In the above steps S101 to S104, by collecting the live images of the road environment, identifying the characters in the live images and extracting the actual first stroke features and the theoretical second stroke features, determining the degree of stroke continuity of the first stroke features relative to the second stroke features, and based on the degree of stroke continuity of the first stroke features and the actual shooting distance, the visibility of the road environment is deduced. By simulating the human eye vision to evaluate the environmental visibility, the problem of incorrect light source recognition can be avoided. Therefore, the accuracy of environmental visibility evaluation can be improved, and it is more in line with the human eye usage habit.

[0082] In some embodiments, the characters can be recognized only by the OCR recognition method, or only by the skeleton recognition method, or both the OCR recognition method and the skeleton recognition method can be used to recognize the characters. The following will illustrate these different situations.

[0083] (1) OCR recognition method:

[0084] In one embodiment, Figure 5 a flowchart of the OCR recognition method is provided, as Figure 5 shown, and the OCR recognition of the live image includes the following steps:

[0085] Step S201, in the live image, determine the first region where the color does not change and is continuous;

[0086] Step S202, perform shape recognition on the first region or the second region obtained by combining the first regions, and determine the corresponding characters according to the recognized shape.

[0087] For easy understanding, Figure 6 a schematic diagram of the live image is provided, as Figure 6 shown. Taking a road sign as an example, assuming that the character "庆" exists in the live image, it can be found that the colors inside the radicals "广" and "大" of this character are white and continuous, while the color between "广" and "大" is white but not continuous. Then, "广" and "大" can be regarded as the first regions respectively, and then these two first regions are combined to obtain the second region. By performing shape recognition on the second region, the corresponding characters can be recognized. Of course, some characters have only the first region and do not need to be combined to obtain the second region. For example, "上" and "了", then only the shape recognition of the first region is required.

[0088] Taking the image recognition of digital information and letter information as an example, based on the coordinates of the color convolution result, the image region is first divided. The regions where the color does not change and is continuous are divided into one region, and the regions where the color does not change but is not continuous are divided into a module (i.e., the second region). Subsequently, optical character recognition is performed using the image shape to complete the recognition of semantic characters such as road signs and speed limit signs.

[0089] (2) Bone recognition method:

[0090] In one embodiment, Figure 7 A flowchart of a bone recognition method is provided, as Figure 7 shown, for performing bone recognition on a live image, including the following steps:

[0091] Step S301, identify pixel blocks with color jumps in the live image.

[0092] Figure 8 A schematic diagram of character recognition is provided, as Figure 8 shown. Taking the character "田" as an example, through the region of interest recognition method, the target region ("田") in the live image can be determined, that is, the image determined by the pixels [x1, y1] to [x2, y2]. Then, in the RGB channel map of the live image, according to the x coordinate from x1 to x2 and the y coordinate from y1 to y2, each color jump pixel block is recognized using an operator of a specific size. Among them, the operator of a specific size refers to a matrix with specific rows and columns, and the matrix size is determined according to the size of the image after cropping processing. For example, the cropped live image can be evenly divided in terms of size along the X-axis and Y-axis directions, and the number of equal divisions can be set by the vehicle-mounted electronic terminal.

[0093] Step S302, determine the median coordinates according to the maximum and minimum values of the pixel coordinates in the pixel block on the X-axis and Y-axis respectively, and determine the pixels at the median coordinates as inflection points.

[0094] An inflection point can refer to the starting and ending points of a certain region in the image. Or, starting from a certain bone point in the image and extending in the four directions of the X-axis and Y-axis of the image, if there are two other bone points with different connection vector directions from this bone point, then this bone point is determined as an inflection point. Among them, a bone point refers to a key point obtained after skeletonizing the object of interest in the image, and the key point is located on the center line of the object of interest and is used to reflect the basic shape and structural characteristics of the object of interest. For Chinese characters, the inflection point is the point of change, starting point, and ending point of its stroke direction; for numbers, the inflection point is the boundary point, starting point, and ending point of its stroke.

[0095] Specifically, referring to Figure 8 , obtain the maximum and minimum values of the coordinates of each color jump pixel block in the X-axis and Y-axis directions, and perform median processing on the maximum and minimum values of the coordinates to obtain the inflection point.

[0096] Taking the character "田" as an example, the calculation is as follows:

[0097] The midpoint of (x1, y1) and (x3, y3) is A[(x1 + x3) / 2, (y1 + y3) / 2];

[0098] The midpoint of (x2, y1), (x6, y3) is B[(x2+x6) / 2, (y1+y3) / 2];

[0099] The midpoint of (x1, y2), (x3, y6) is C[(x1+x3) / 2, (y2+y6) / 2];

[0100] The midpoint of (x6, y6), (x2, y2) is D[(x2+x6) / 2, (y6+y2) / 2];

[0101] The midpoint of (x3, y4), (x3, y5) is E[(x3+x3) / 2, (y4+y5) / 2];

[0102] The midpoint of (x4, y3), (x5, y3) is F[(x4+x5) / 2, (y3+y3) / 2];

[0103] The midpoint of (x4, y6), (x5, y6) is G[(x4+x5) / 2, (y6+y6) / 2];

[0104] The midpoint of (x6, y4), (x6, y5) is H[(x6+x6) / 2, (y4+y5) / 2].

[0105] Among them, the midpoints of E and H, F and G are all O, A, B, D, and C are image boundary points (also inflection points), and E, F, G, H, and O are all inflection points.

[0106] Step S303, sequentially connect every two inflection points that are closest to each other to obtain skeleton information; and determine the corresponding character according to the skeleton information.

[0107] Determine every two pairs of inflection points that are closest to each other, perform vector operations on each pair of inflection points, and form a first stroke order matrix; compare the first stroke order matrix with a preset second stroke order matrix, and determine corresponding characters according to the similarity between the two; wherein the second stroke order matrix is ​​associated with the corresponding characters. After comparing the first stroke order matrix with the preset second stroke order matrix and determining corresponding characters according to the similarity between the two, add the distances between the two inflection points in each pair of inflection points corresponding to the first stroke order matrix to obtain the first stroke feature of the character; add the distances between the two inflection points in each pair of inflection points corresponding to the second stroke order matrix to obtain the second stroke feature.

[0108] In practical applications, for a live image after region of interest recognition and cropping, starting from the upper left corner of the live image, the nearest inflection point can be sought for each inflection point, and vector operations are performed point by point for each inflection point, so that the entire skeleton route can be obtained, which is the first stroke order matrix. Based on the same principle as the above method, skeleton recognition is performed for a clear live image, and the second stroke order matrix is ​​disassembled and obtained, and multiple second stroke order matrices are pre-stored to obtain a standard stroke order matrix library. The standard stroke order matrix library is traversed, and each first stroke order matrix is ​​compared with the second stroke order matrix, and the character associated with the second stroke order matrix with the highest similarity is taken as the recognized character.

[0109] (3) OCR recognition method combined with skeleton recognition method:

[0110] In one embodiment, Fig. 9 A flowchart of combining OCR recognition method with skeleton recognition method is provided, such as Fig. 9 As shown, the following steps are included:

[0111] Step S401, perform OCR recognition on the live image to obtain an OCR recognition result; for details, please refer to Figure 5 Steps shown;

[0112] Step S402, performing skeleton recognition on the live image to obtain a skeleton recognition result; Figure 7 Steps shown;

[0113] Step S403, determining whether the OCR recognition result and the skeleton recognition result are consistent; if they are consistent, executing step S404; if they are inconsistent, executing step S405;

[0114] Step S404, determining that the current semantic recognition process is effective, and outputting corresponding character information;

[0115] Step S405: determine that the current semantic recognition process is invalid, and re-perform the semantic recognition process based on the next frame of live image.

[0116] In the present embodiment, the characters identified by the skeleton recognition method are compared with the characters identified by the OCR recognition method. When the recognition results of the two algorithms are the same, it is determined that the recognition is valid and the character information is output. The character information can be the character itself (classical Chinese, Arabic numerals, symbols), or the boundary point of the character, wherein the boundary point refers to the coordinates of the stroke boundary points of a complete character. When the recognition results of the two algorithms are different, the semantic recognition will use the next frame of live image to re-recognize until the correct character information is output, or the target object in the road environment moves out of the visual field of the camera projection. This arrangement is to take into account that the skeleton recognition method performs better when processing complex backgrounds and partial occlusions, while the OCR recognition method is more accurate when processing clear and standard characters. When the recognition results of the two algorithms are consistent, it can be considered that the recognition results are reliable, thereby outputting character information, and this dual verification mechanism significantly reduces the probability of misidentification. If the results of the two algorithms are inconsistent, the next frame of image will continue to be used for re-recognition until the results of the two methods are consistent or the target object moves out of the visual field, and this dynamic adjustment and continuous verification process further improves the accuracy of recognition. In practical applications, characters in road environments may be affected by a variety of factors, such as lighting changes, occlusion, and contamination. The multi-algorithm fusion method can better cope with these complex situations and improve the robustness of the system. This embodiment significantly improves the accuracy of semantic recognition through the design of multi-algorithm fusion, complementarity, double verification, robustness, and real-time performance, ensuring that character information can be reliably recognized in various complex environments.

[0117] In one embodiment, Fig.10 A flowchart for preprocessing live images is provided, such as Fig.10 As shown, before performing semantic recognition processing on the live image, the method also includes the following steps:

[0118] Step S501, grayscale processing is performed on the live image according to the three channels of R, G, and B to obtain a first grayscale image of the live image on the three channels of R, G, and B.

[0119] The collected live image can be grayscaled in the three color layers of R (red), G (green) and B (blue) to obtain the grayscale distribution of the live image in the three color layers. For the white area with green as the background color in the live image, the grayscale will show a highlight color in the three RGB image layers.

[0120] refer to Figure 6, taking road signs as target objects, in the actual road test process, a large number of real-time images containing target objects can be collected as sample images for model training, and the color information of the target objects in the sample images can be extracted. For example: when green represents the background, the text, numbers, destination distance, direction and other information are white; or when blue represents the background, the text, numbers, destination distance, direction and other information are white.

[0121] Step S502: performing convolution processing on the first grayscale image to obtain a second grayscale image.

[0122] For example, on the green layer, a certain size of graphic window is used for convolution operation. Since the grayscale value of green pixels in the G layer is high, the high grayscale value area in the second grayscale image after convolution is the green road sign part, thus obtaining the pixel coordinates [x1, y1] to [x2, y2] of the road sign in the live image. Since the grayscale values ​​of the white text and numbers in the green area are consistent in the RGB layer (the position characteristics of white in the grayscale division are consistent), similar coordinate positions can be obtained in the three color layers. Similarly, for target objects of other colors, the same color determination method can be used to obtain the coordinate position of the target object in the image.

[0123] Step S503, determining a target area in the second grayscale image whose pixel values ​​are higher than a preset threshold, and cropping the real-life image to retain the target area; wherein the cropped real-life image is used as an input image for semantic recognition processing.

[0124] Fig.11 is a schematic diagram of the cropped live image, such as Fig.11 As shown, based on the coordinate information of the target object that has been obtained by the convolution layer, the original image size is retained according to [x1, y1] ~ [x2, y2] to form a new image, which is passed to the downstream (semantic recognition) for the next step of processing. In practical applications, there are usually multiple characters in an original live image, so multiple sub-images are cut for each character. The continuity of the stroke features of the multiple sub-images can be combined to evaluate the environmental visibility. Specifically, the minimum environmental visibility evaluation value obtained by measurement can be taken as the current environmental visibility, or the average of the multiple environmental visibility evaluation values ​​obtained by measurement can be taken as the current environmental visibility. This embodiment is not limited.

[0125] This embodiment separates the original image by identifying the color features of the target object, thereby reducing the image size while retaining the information of the target object, which is beneficial to reducing the amount of calculation for subsequent semantic recognition processing.

[0126] In one embodiment, obtaining a preset data set includes: setting scenes obtained by combining multiple different distance gradients and visibility gradients; collecting live images in each scene, and extracting stroke features of characters in the live images; wherein the stroke features include a third stroke feature and a fourth stroke feature, and the third stroke feature corresponds to a scene with a highest visibility gradient; using the third stroke feature as a reference object, determining a stroke continuity degree of the fourth stroke feature relative to the third stroke feature in each scene; and constructing a preset data set according to the shooting distance of each scene, the environmental visibility, and the calculated stroke continuity degree of the fourth stroke feature.

[0127] In this embodiment, the stroke continuity of the vehicle under different distance gradients and visibility gradients can be calibrated in advance. Exemplarily, nine different test scenes can be cross-constructed in the test field according to the two dimensions of distance (far, medium, near) and visibility (low, medium, high) to calibrate the vehicle's recognition ability of the target object. In each scene, the target object is photographed by the vehicle camera, and the target recognition is performed for the images under different visibility conditions, especially the characters on the target object. For scenes with high visibility, the total length L1 of the recognized character strokes is calculated; similarly, the above process is repeated under the conditions of medium and low visibility, and the total lengths L2 and L3 of the character strokes are calculated respectively. By comparing the ratios of L2 / L1 and L3 / L1, the stroke continuity Q under different visibilities can be evaluated. In this way, a coincidence matrix [distance, stroke continuity, environmental visibility] containing distance, stroke continuity and environmental visibility can be generated to analyze the recognition effect of the vehicle on the target object under different conditions.

[0128] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0129] Obtain the real-time image of the road environment captured by the camera, as well as the actual shooting distance of the camera;

[0130] Recognize characters in a live image, extract first stroke features of the characters, and extract second stroke features of the characters in full size;

[0131] Taking the second stroke feature as a reference object, determining the stroke continuity degree of the first stroke feature relative to the second stroke feature;

[0132] The visibility of the road environment is determined according to the stroke continuity of the first stroke feature and the actual shooting distance.

[0133] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0134] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0135] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for evaluating environmental visibility, characterized in that: include: Acquire a real-time image of the road environment captured by a camera, and an actual shooting distance of the camera; Recognize characters in the live image, extract first stroke features of the characters, and extract second stroke features of the characters in full size; Taking the second stroke feature as a reference object, determining the stroke continuity degree of the first stroke feature relative to the second stroke feature; The visibility of the road environment is determined according to the stroke continuity of the first stroke feature and the actual shooting distance.

2. The method for evaluating environmental visibility according to claim 1, characterized in that: Recognizing characters in the live image, extracting first stroke features of the characters, and extracting second stroke features of the characters in full size, comprising: Performing semantic recognition processing on the live image to identify the characters; wherein the semantic recognition processing includes OCR recognition and / or skeleton recognition; The first stroke feature and the second stroke feature are obtained according to the character extraction.

3. The method for evaluating environmental visibility according to claim 2, characterized in that: Performing OCR recognition on the live image includes: In the live image, determining a first region whose color does not change and is continuous; Shape recognition is performed on the first area or the second area obtained by combining the first areas, and a corresponding character is determined according to the recognized shape.

4. The method for evaluating environmental visibility according to claim 2, characterized in that: Performing skeleton recognition on the live image includes: Identifying pixel blocks where color jump occurs in the live image; Determine median coordinates according to the maximum and minimum values ​​of pixel coordinates in the pixel block on the X-axis and the Y-axis, respectively, and determine the pixel at the median coordinates as an inflection point; Connecting every two inflection points that are closest to each other in sequence to obtain skeleton information; The corresponding character is determined according to the skeleton information.

5. The method for evaluating environmental visibility according to claim 4, characterized in that: Connecting every two inflection points that are closest to each other in sequence to obtain skeleton information; Determining a corresponding character according to the skeleton information includes: Determine every two pairs of inflection points that are closest to each other, perform vector operations on each pair of inflection points, and form a first stroke order matrix; The first stroke order matrix is ​​compared with a preset second stroke order matrix, and corresponding characters are determined according to the similarity between the two; wherein the second stroke order matrix is ​​associated with corresponding characters.

6. The method for evaluating environmental visibility according to claim 5, characterized in that: After comparing the first stroke order matrix with a preset second stroke order matrix and determining corresponding characters according to the similarity between the two, the method further includes: Adding the distances between two inflection points in each inflection point pair corresponding to the first stroke order matrix to obtain a first stroke feature of the character; The distances between two inflection points in each inflection point pair corresponding to the second stroke order matrix are added to obtain the second stroke feature.

7. The method for evaluating environmental visibility according to claim 2, characterized in that: Performing semantic recognition processing on the live image to recognize the characters includes: Performing OCR recognition and skeleton recognition on the live image to obtain an OCR recognition result and a skeleton recognition result respectively; Determine whether the OCR recognition result is consistent with the skeleton recognition result; If the OCR recognition result is consistent with the skeleton recognition result, it is determined that the current semantic recognition process is valid, and the corresponding character information is output; If the OCR recognition result is inconsistent with the skeleton recognition result, it is determined that the current semantic recognition process is invalid, and the semantic recognition process is performed again based on the next frame of live image.

8. The method for evaluating environmental visibility according to claim 2, characterized in that: Before performing semantic recognition processing on the live image, the method further includes: Performing grayscale processing on the live image according to the three channels of R, G, and B to obtain a first grayscale image of the live image on the three channels of R, G, and B; Performing convolution processing on the first grayscale image to obtain a second grayscale image; Determine a target area in the second grayscale image whose pixel values ​​are higher than a preset threshold, and crop the live image to retain the target area; wherein the cropped live image is used as an input image for the semantic recognition process.

9. The environmental visibility assessment method according to claim 1, characterized in that: Determining the visibility of the road environment according to the stroke continuity of the first stroke feature and the actual shooting distance includes: Acquire a preset data set, wherein the preset data set includes a plurality of reference data, each of which includes mutually related shooting distances, stroke continuity, and environmental visibility; The information formed by the stroke continuity of the first stroke feature and the actual shooting distance is compared with a preset data set, and the visibility of the current road environment is determined according to the comparison result.

10. The method for evaluating environmental visibility according to claim 9, characterized in that: Get preset data sets, including: Set up scenes with multiple combinations of distance gradients and visibility gradients; Collecting live images in each of the scenes, and extracting stroke features of characters in the live images; wherein the stroke features include a third stroke feature and a fourth stroke feature, and the third stroke feature corresponds to a scene with the highest visibility gradient; Taking the third stroke feature as a reference object, determining the stroke continuity degree of the fourth stroke feature relative to the third stroke feature in each scene; The preset data set is constructed according to the shooting distance of each scene, the environmental visibility and the calculated stroke continuity of the fourth stroke feature.

11. A vehicle light control method, characterized in that: include: Determine the visibility of the current road environment according to the environmental visibility assessment method according to any one of claims 1 to 10; Control the operating status of the vehicle headlights according to the visibility of the current road environment.

12. An in-vehicle electronic device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

Images