Image display method and vehicle

Abstract

This application provides an image display method and a vehicle. The method includes: acquiring a current frame night vision image captured by a night vision camera unit; determining at least one image region to be tested from the current frame night vision image; extracting image feature vectors of each image region to be tested and inputting them into a pre-trained classifier to obtain a first classification result indicating whether a pedestrian exists in the image region to be tested; if no first classification result is obtained, extracting image sub-feature vectors of each image sub-region in the image region to be tested and determining a second classification result for the image region to be tested; based on the first or second classification result corresponding to each image region to be tested, marking the regions where pedestrians are determined to exist in the current frame night vision image to obtain a target night vision image; and displaying the target night vision image in a display unit. The technical solution provided by the embodiments of this application can improve the reliability of pedestrian detection in night vision images.

Description

Technical Field

[0001] This application relates to the field of image display technology, and more specifically, to an image display method and a vehicle. Background Technology

[0002] While existing vehicle-mounted night vision systems improve nighttime driving visibility, their core pedestrian detection function faces reliability challenges in complex scenarios. For example, in low light or foggy weather, image quality deteriorates significantly, making feature extraction difficult with traditional detection algorithms, thus significantly reducing the accuracy of pedestrian identification. Furthermore, when pedestrians are partially obscured by vehicles, trees, or other objects, the incomplete overall features of the pedestrian lead to missed detections using traditional algorithms, creating serious blind spots. These issues severely compromise the safety assistance capabilities of night vision systems, failing to provide drivers with stable and reliable pedestrian warnings. Summary of the Invention

[0003] The embodiments of this application provide an image display method and a vehicle. Based on the technical solution provided in this application, the reliability of pedestrian detection for night view images can be improved.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to a first aspect of the embodiments of this application, an image display method is provided, applied to a vehicle equipped with a night vision camera unit and a display unit. The method includes: acquiring a current frame night vision image captured by the night vision camera unit; determining at least one image region to be tested from the current frame night vision image, each image region to be tested consisting of multiple image sub-regions; for each image region to be tested, extracting the image feature vector of the image region to be tested, and inputting the image feature vector into a pre-trained classifier to obtain a first classification result of whether a pedestrian exists in the image region to be tested; if no first classification result is obtained for the image region to be tested, extracting the image sub-feature vector of each image sub-region in the image region to be tested; determining a second classification result of whether a pedestrian exists in the image region to be tested based on the image sub-feature vector of each image sub-region; marking the areas where pedestrians are determined to exist in the current frame night vision image based on the first classification result or the second classification result corresponding to each image region to be tested, thereby obtaining a target night vision image; and displaying the target night vision image in the display unit.

[0006] In some embodiments of this application, based on the aforementioned scheme, the vehicle is further equipped with a panoramic camera unit. Before acquiring the current frame night view image captured by the night vision camera unit, the method further includes: acquiring the current state parameters of the panoramic camera unit and the night vision camera unit, as well as the environmental parameters of the current environment; if the current state parameters and environmental parameters meet a first preset condition, then the step of acquiring the current frame night view image captured by the night vision camera unit is executed; if the current state parameters and environmental parameters do not meet the first preset condition, then the current frame panoramic image captured by the panoramic camera unit is acquired, and the current frame panoramic image is displayed in the display unit.

[0007] In some embodiments of this application, based on the aforementioned scheme, the image feature vector of the image region to be tested is extracted, including: extracting the image sub-feature vector of each image sub-region in the image region to be tested; concatenating the image sub-feature vectors of each image sub-region in the image region to be tested according to a preset rule to obtain a concatenated vector; and using the transpose of the concatenated vector as the image feature vector of the image region to be tested.

[0008] In some embodiments of this application, based on the aforementioned scheme, the image feature vector is input to a pre-trained classifier to obtain a first classification result of whether there is a pedestrian in the image region to be tested, including: inputting the image feature vector to the classifier so that the classifier can calculate the classification confidence score of the image region to be tested using the following decision function; H(x) = ε + w T ×x Where H(x) represents the classification confidence score; ε represents the bias term of the classifier; w T denoted by , where x represents the transpose of the classifier's weight vector; x represents the image feature vector; if the classification confidence score is greater than the first threshold, a first classification result indicating the presence of a pedestrian is obtained; if the classification confidence score is less than the second threshold, a first classification result indicating the absence of a pedestrian is obtained, where the second threshold is less than the first threshold; if the classification confidence score is between the first and second thresholds, it is determined that a first classification result for the image region under test has not been obtained.

[0009] In some embodiments of this application, based on the aforementioned scheme, determining whether there is a pedestrian in the image region to be tested based on the image sub-feature vector of each image sub-region includes: for each image sub-region in the image region to be tested, extracting the weight sub-vector corresponding to the image sub-region from the weight vector of the classifier; and calculating the classification confidence sub-score corresponding to the image sub-region according to the following formula. h(i)=ε i +w i T ×C i Where h(i) represents the classification confidence sub-score, ε i This represents the sub-bias term corresponding to the sub-region of the image; w i T C represents the transpose of the weight sub-vector corresponding to the sub-region of the image; i The image sub-feature vector represents the sub-region of the image; based on the classification confidence sub-score corresponding to each sub-region of the image region to be tested, a second classification result is determined to determine whether there is a pedestrian in the image region to be tested.

[0010] In some embodiments of this application, based on the aforementioned scheme, a second classification result is determined regarding whether a pedestrian exists in the image region under test, based on the classification confidence sub-score corresponding to each image sub-region in the image region under test. This includes: for each image sub-region in the image region under test, comparing the classification confidence sub-score corresponding to the image sub-region with a preset decision threshold; if the classification confidence sub-score of the image sub-region is greater than the decision threshold, assigning a first-class label indicating the presence of a pedestrian to the image sub-region; if the classification confidence sub-score of the image sub-region is not greater than the decision threshold, assigning a second-class label indicating the absence of a pedestrian to the image sub-region; and determining the second classification result regarding whether a pedestrian exists in the image region under test based on the distribution of the first-class and second-class labels in the image region under test.

[0011] In some embodiments of this application, based on the aforementioned scheme and the distribution of the first and second type labels in the image region to be tested, a second classification result indicating whether a pedestrian exists in the image region to be tested is determined, including: using a mean-shift segmentation algorithm to optimize the initial label distribution of the first and second type labels in the image region to be tested to obtain an optimized label distribution; if the proportion of image sub-regions labeled with the first type label in the optimized label distribution is greater than a preset proportion, then the image region to be tested is determined to be a second classification result indicating the presence of a pedestrian.

[0012] In some embodiments of this application, based on the foregoing scheme, the method further includes: acquiring a color training image set; performing grayscale processing on each color training image in the color training image set to obtain a grayscale training image set; performing exponential transformation processing on each grayscale training image in the grayscale training image set to obtain a sample training image set simulating a night vision environment; extracting the sample feature vector of each sample training image in the sample training image set to obtain a sample feature vector set; and training a pre-constructed support vector machine model based on the feature vector set to obtain a classifier.

[0013] In some embodiments of this application, based on the aforementioned scheme, displaying a target night view image in a display unit includes: scaling and cropping the target night view image to obtain a new target night view image adapted to the display window of the display unit; and displaying the new target night view image in the display window.

[0014] According to a second aspect of the embodiments of this application, a vehicle is provided, the vehicle being configured with a night vision camera unit and a display unit, the vehicle further comprising one or more processors and one or more memories, the one or more memories storing at least one piece of program code, the at least one piece of program code being loaded and executed by the one or more processors to perform the operations performed as described in any of the first aspects above.

[0015] The technical solution of this application processes the night vision image to be displayed according to the following steps before displaying it: First, acquire the current frame night vision image captured by the night vision camera unit; second, determine at least one test image region from the current frame night vision image, each test image region consisting of multiple image sub-regions; third, for each test image region, extract the image feature vector of the test image region and input the image feature vector into a pre-trained classifier to obtain a first classification result of whether a pedestrian exists in the test image region; if no first classification result is obtained for the test image region, extract the image sub-feature vector of each image sub-region in the test image region; fourth, based on the image sub-feature vector of each image sub-region, determine a second classification result of whether a pedestrian exists in the test image region; fifth, based on the first or second classification result corresponding to each test image region, mark the regions in the current frame night vision image where pedestrians are determined to exist, to obtain the target night vision image; finally, display the target night vision image in the display unit.

[0016] Therefore, the technical solution provided in this application does not rely on a single classifier for pedestrian detection. Instead, it first uses a trained classifier to quickly make a preliminary judgment on the complete image region to be tested, obtaining a first classification result. For cases where the judgment result is unclear or a reliable first classification result indicating the presence or absence of a pedestrian is not obtained, the detection is not terminated directly, and no detection result is output, or a detection result indicating the absence of a pedestrian is output. Instead, a more refined local judgment mechanism is initiated. In this local judgment mechanism, each sub-region of the image region to be tested is analyzed and judged, thereby determining an accurate second classification result for the image region to be tested. Based on this pedestrian detection scheme, the false negative rate and false positive rate of pedestrians in night vision images can be significantly reduced, thereby greatly improving the reliability of pedestrian detection in night vision images and providing drivers with stable and reliable pedestrian warnings.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 This illustration shows a detailed flowchart of whether to acquire the current frame night vision image captured by the night vision camera unit according to an embodiment of the present application; Figure 2 A flowchart illustrating an image display method according to an embodiment of this application is shown; Figure 3 The diagram illustrates a detailed process according to an embodiment of the present application, in which an image feature vector is input into a pre-trained classifier to obtain a first classification result indicating whether a pedestrian exists in the region of the image to be tested. Figure 4 This illustration shows a detailed flowchart of a second classification result based on the image sub-feature vector of each image sub-region according to an embodiment of the present application, to determine whether a pedestrian exists in the image region to be tested. Figure 5 This illustration shows a detailed flowchart of a second classification result based on the classification confidence sub-score corresponding to each image sub-region in the image region to be tested, according to an embodiment of the present application, to determine whether there is a pedestrian in the image region to be tested; Figure 6 A detailed flowchart illustrating the display of a target night view image in a display unit according to an embodiment of this application is shown. Figure 7 A schematic diagram of the structure of a vehicle according to an embodiment of this application is shown. Detailed Implementation

[0019] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0020] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0021] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0022] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0023] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such uses of these terms can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described.

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0026] It should be noted that the technical solution of this application is applied to a vehicle equipped with a night vision camera unit and a display unit. The night vision camera unit can capture night vision images, and the display unit can display the image information of the night vision images captured by the night vision camera unit to the driver.

[0027] In some implementations, the vehicle is also equipped with a panoramic camera unit that can capture panoramic image information. Understandably, night vision camera units, using specialized cameras such as thermal imaging or near-infrared sensors, enable drivers to see pedestrians, animals, or objects that are difficult to detect with the naked eye in extremely dark or no-light environments, thus improving driving safety at night or in inclement weather. Panoramic camera units typically use multiple standard cameras stitched together to present the driver with a color view of the overall surrounding environment on a display unit.

[0028] In some implementations, the vehicle is also equipped with an environmental sensor that can detect environmental information about the vehicle's current environment, such as whether there is extreme weather such as heavy fog.

[0029] The following detailed description of some embodiments of this application will be provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0030] See Figure 1 This document illustrates a detailed flowchart of whether or not the night vision image of the current frame acquired by the night vision camera unit is acquired according to an embodiment of this application, specifically including the following steps 110 to 130: Step 110: Obtain the current state parameters of the panoramic camera unit and the night vision camera unit, as well as the environmental parameters of the current environment.

[0031] In this embodiment, the current status parameters of the panoramic camera unit and the night vision camera unit include, but are not limited to: the current fault state one of the panoramic camera and the current fault state two of the night vision camera unit; the current time parameter; the current temperature parameter of the night vision camera unit, etc.

[0032] In this embodiment, the environmental parameters of the current environment of the panoramic camera unit and the night vision camera unit include, but are not limited to: ambient brightness parameters; ambient visibility parameters, etc.

[0033] It is understandable that this environmental parameter can be obtained based on the environmental sensors configured in the vehicle.

[0034] See also Figure 1 Step 120: If the current state parameters and environmental parameters meet the first preset condition, then execute the step of acquiring the current frame night vision image collected by the night vision camera unit.

[0035] In this embodiment, the first preset condition can be that current fault state one indicates that the panoramic camera unit is faulty, and current fault state two indicates that the night vision camera unit is not faulty. That is, if the panoramic camera unit is faulty and the night vision camera unit is not faulty, then the step of acquiring the current frame night vision image captured by the night vision camera unit is executed.

[0036] Understandably, if both the panoramic camera unit and the night vision camera unit malfunction, the vehicle's image display function can be turned off directly.

[0037] In this embodiment, the first preset condition can also be that the current time is within a preset time period, the environmental visibility parameter is lower than the preset visibility, and the current temperature parameter of the night vision camera unit is lower than the preset temperature threshold. The preset time period indicates that the current time is nighttime; therefore, the time period from 7:00 PM to 6:00 AM the next day can be set as this preset time period. Currently, this preset time period can also be set according to actual conditions. The environmental visibility parameter being lower than the preset visibility indicates that the current environment is experiencing extreme weather conditions such as heavy fog; therefore, this preset visibility can be set according to actual conditions, and this application does not impose any limitations on it.

[0038] In this embodiment, the first preset condition can also be that the ambient brightness parameter in the environmental parameters is less than a preset brightness threshold, the ambient visibility parameter in the environmental parameters is lower than a preset visibility, and the current temperature parameter of the night vision camera unit is less than a preset temperature threshold. Wherein, the ambient brightness parameter being less than the preset brightness threshold indicates that the current ambient brightness is very low and the driver's visibility is very poor.

[0039] See also Figure 1 Step 130: If the current state parameters and environmental parameters do not meet the first preset condition, then acquire the current frame panoramic image captured by the panoramic camera unit and display the current frame panoramic image in the display unit.

[0040] In this embodiment, if the current time is not within a preset time period, i.e., the current time is during the day, then the current frame panoramic image is displayed in the display unit.

[0041] In this embodiment, if the environmental visibility parameter in the environmental parameters is higher than the preset visibility, then the current frame panoramic image is displayed in the display unit. If the environmental visibility parameter in the environmental parameters is higher than the preset visibility, it indicates that it is currently daytime.

[0042] In this embodiment, if the current temperature parameter of the night vision camera unit is higher than a preset temperature threshold, the current frame panoramic image can be displayed in the display unit.

[0043] In this application, by comprehensively judging the status and environmental parameters of the panoramic camera unit and the night vision camera unit, the system intelligently decides whether to enable night vision or panoramic imagery. This avoids the system from activating night vision in inappropriate scenarios (such as during the day when the weather is good) and causing inefficient operation. It also avoids the loss of image support when the night vision camera unit malfunctions (such as when the camera overheats). Therefore, based on the technical solution provided in this application, the adaptability and robustness of the vehicle system can be improved.

[0044] In some technical solutions of this application, the specific implementation methods for displaying the acquired night view images can be referred to as follows: Figure 2 Perform the steps shown: See Figure 2 The diagram illustrates a flowchart of an image display method according to an embodiment of this application, specifically including steps 210 to 270: Step 210: Obtain the current frame night vision image captured by the night vision camera unit.

[0045] See also Figure 2 Step 220: Determine at least one image region to be tested from the current frame night view image, each image region to be tested consisting of multiple image sub-regions.

[0046] In this embodiment, a sliding window method can be used to determine each image region to be tested in the current frame night view image. Specifically, a detection window of a fixed size can be set, and starting from the upper left corner of the current frame night view image, it can be slid across all positions of the current frame night view image with a certain step size, thereby determining an image region to be tested at each window position.

[0047] In some implementations, each image region to be tested can be divided into 55 image sub-regions, with each sub-region being 1px × 1px.

[0048] See also Figure 2 Step 230: For each image region to be tested, extract the image feature vector of the image region to be tested, and input the image feature vector into a pre-trained classifier to obtain the first classification result of whether there is a pedestrian in the image region to be tested.

[0049] In some implementations, the specific method for extracting the image feature vector of the image region to be tested can be performed according to the following steps: Extract the image sub-feature vector of each image sub-region in the image region to be tested; The image sub-feature vectors of each image sub-region in the image region to be tested are concatenated according to preset rules to obtain a concatenated vector; The transpose of the spliced ​​vector is used as the image feature vector of the image region to be tested.

[0050] Specifically, the image sub-feature vector of each image sub-region in the image region to be tested can be 32-dimensional. It can be obtained by calculating the Histogram of Oriented Gradients (HOG) features of each image sub-region, denoted as C. i .

[0051] Furthermore, assuming the image region to be tested comprises 55 image sub-regions, the concatenated vector obtained by concatenating the image sub-feature vectors of the 55 image sub-regions can be represented as Y = (C1, C2, ..., C...). 55 ).

[0052] Furthermore, the image feature vector of the region to be tested can be represented as: x = (C1, C2, ..., C 55 ) T .

[0053] In this embodiment, by first extracting the image sub-feature vectors and then concatenating them to generate the image feature vector of the entire image to be tested, a structured and effective input containing spatial context information is provided for the subsequent classifier, thereby improving the representation ability of the image feature vector.

[0054] In step 230, the image feature vector is input into a pre-trained classifier to obtain a first classification result indicating whether a pedestrian exists in the region of the image to be tested. For a detailed implementation of this method, please refer to [link / reference needed]. Figure 3 Perform the steps shown.

[0055] See Figure 3 This diagram illustrates a detailed process according to an embodiment of the present application, in which an image feature vector is input into a pre-trained classifier to obtain a first classification result indicating whether a pedestrian exists in the region of the image to be tested. Specifically, it includes steps 231 to 234: Step 231: Input the image feature vector into the classifier so that the classifier can calculate the classification confidence score of the image region to be tested using the decision function of the following formula 1. H(x) = ε + w T ×xFormula 1 Where H(x) represents the classification confidence score; ε represents the bias term of the classifier; w T represents the transpose of the classifier's weight vector; x represents the image feature vector.

[0056] In some embodiments of this application, the classifier can be trained using the following steps: Obtain a set of color training images; convert each color training image in the set to grayscale to obtain a set of grayscale training images; perform exponential transformation on each grayscale training image in the set to obtain a set of sample training images simulating night vision conditions; extract the sample feature vector of each sample training image in the set to obtain a set of sample feature vectors; train a pre-built support vector machine model based on the feature vector set to obtain a classifier.

[0057] The color training image set includes multiple color training images, each captured during the day, and each containing both pedestrian and non-pedestrian areas. After acquiring the color training image set, it can be labeled to indicate whether pedestrians are present or not.

[0058] Next, the color image training set is converted to grayscale to transform the color images into single-channel grayscale images, i.e., images without color information. It can be understood that the grayscale training image set includes multiple grayscale training images that correspond one-to-one with the color training images.

[0059] Next, an exponential transform is performed on the grayscale training image set to adjust the brightness and contrast of each grayscale training image in the set, simulating the visual characteristics under night vision conditions. This results in a sample training image set simulating night vision conditions.

[0060] Understandably, the support vector algorithm is used during the training of the support vector machine, resulting in a classifier that includes the two parameters required in Equation 1: a bias term and a weight vector. The classifier's bias term ε characterizes the inherent tendency or baseline value of the classifier's decision function when there are no feature inputs (i.e., the image feature vector x is zero). This is essentially the offset of the decision hyperplane in the feature space.

[0061] In this context, the weight coefficients W in the classifier characterize the relative importance or contribution of each feature learned by the classifier to the final classification decision. It can also be expressed as the weighted sum of all support vectors learned by the classifier.

[0062] See also Figure 3 Step 232: If the classification confidence score is greater than the first threshold, the first classification result indicating the presence of a pedestrian is obtained.

[0063] Understandably, a higher classification confidence score indicates a higher probability that a pedestrian exists in the region of the image being tested. Theoretically, a higher H(x) value indicates a higher probability that a pedestrian exists in the region of the image being tested, and vice versa. If the classifier's response is close to 0, the classification is considered fuzzy.

[0064] In some implementations, the first threshold can be set to 0.5. Therefore, if the obtained H(x) is greater than 0.5, it can be directly determined that there is a pedestrian in the image region to be tested, and a first classification result is obtained to indicate that there is a pedestrian in the image region to be tested.

[0065] See also Figure 3 Step 233: If the classification confidence score is less than the second threshold, then the first classification result indicating that there is no pedestrian is obtained, and the second threshold is less than the first threshold.

[0066] In some implementations, the second threshold can be set to -1.5. Therefore, if the obtained H(x) is less than -1.5, it can be directly determined that there are no pedestrians in the image region to be tested, that is, the image region to be tested is a background region.

[0067] See also Figure 3 In step 234, if the classification confidence score is between the first threshold and the second threshold, it is determined that the first classification result of the image region to be tested has not been obtained.

[0068] Understandably, the obtained H(x) falls within the range of (-1.5, 0.5), indicating that the first classification result for determining whether a pedestrian exists in the image region is ambiguous, suggesting potential occlusion in the image region. Therefore, it is determined that no first classification result was obtained for the image region.

[0069] In this embodiment, by introducing a classification confidence scoring mechanism with a clear threshold range, the classification results are clearly divided into three categories: certain to exist, certain to not exist, and ambiguous and awaiting review. This provides clear and reliable triggering conditions for initiating more complex and detailed analyses, and achieves optimized allocation of computing resources.

[0070] See also Figure 2 Step 240: If the first classification result of the image region to be tested is not obtained, then extract the image sub-feature vector of each image sub-region in the image region to be tested.

[0071] See also Figure 2 Step 250: Based on the image sub-feature vector of each image sub-region, determine whether there is a second classification result for the presence of pedestrians in the image region to be tested.

[0072] In some implementations, the specific implementation of determining whether a pedestrian exists in the image region under test based on the image sub-feature vector of each image sub-region can be referred to as follows: Figure 4 Perform the steps shown: See Figure 4This document illustrates a detailed flowchart of a second classification result based on the image sub-feature vector of each image sub-region, according to an embodiment of this application. The process specifically includes steps 251 to 253. Step 251: For each sub-region of the image region to be tested, extract the weight sub-vector corresponding to the sub-region from the weight vector of the classifier.

[0073] In this embodiment, it can be understood that the weight vector in the classifier has a weight sub-vector corresponding to each image sub-region. Therefore, the weight sub-vector corresponding to the image sub-region can be directly extracted from the weight vector.

[0074] It is also understandable that the weight sub-vectors corresponding to each sub-region of the image are not necessarily the same.

[0075] See also Figure 4 Step 252: Calculate the classification confidence sub-score corresponding to the sub-region of the image according to the following formula 2; h(i)=ε i +w i T ×C i Formula 2 Where h(i) represents the classification confidence sub-score, ε i This represents the sub-bias term corresponding to the sub-region of the image; w i T C represents the transpose of the weight sub-vector corresponding to the sub-region of the image; i The image sub-feature vector represents the sub-region of the image.

[0076] It should be noted that the ε i This means that the classifier learns from the feature vectors of the training images. In other words, during the training of the classifier, a sub-bias term corresponding to each sub-region of the image can be obtained.

[0077] In this embodiment, the larger the value of h(i), the higher the probability that there is a pedestrian in the image sub-region.

[0078] In this embodiment, by deriving weight sub-vectors and sub-bias terms for each image sub-region from the classifier's weight vector, the classification knowledge of the classifier is reused and refined, making the evaluation of local regions consistent with the global model and more targeted, laying the foundation for subsequent processing of occlusion and other situations in the image.

[0079] See also Figure 4Step 253: Based on the classification confidence sub-score corresponding to each image sub-region in the image region to be tested, determine whether there is a second classification result for pedestrians in the image region to be tested.

[0080] In some implementations, the specific implementation of step 253 can be referred to Figure 5 Perform the steps shown: See Figure 5 The diagram illustrates a detailed flowchart of a second classification result based on the classification confidence sub-score corresponding to each image sub-region in the image region to be tested, according to an embodiment of the present application. The process includes the following steps 2531 to 2534: Step 2531: For each image sub-region in the image region to be tested, compare the classification confidence sub-score corresponding to the image sub-region with the preset decision threshold.

[0081] In some implementations, a preset decision threshold can be set to 0.

[0082] See also Figure 5 Step 2532: If the classification confidence score of the image sub-region is greater than the decision threshold, then assign the first-class label of "Pedestrian Existence" to the image sub-region.

[0083] Understandably, if the classification confidence sub-score of the obtained image sub-region is greater than 0, it indicates that there is a pedestrian in the image sub-region. Therefore, the image sub-region is assigned a first-class label indicating the presence of a pedestrian, which can be represented as t. i The first type of label t i The value can be 1.

[0084] See also Figure 5 Step 2533: If the classification confidence score of the image sub-region is not greater than the decision threshold, then assign the second category label of "no pedestrians" to the image sub-region.

[0085] It is understandable that if the classification confidence score of the obtained image sub-region is less than or equal to 0, it means that there is no pedestrian in the image sub-region. It may be due to occlusion or being part of the background. Therefore, the image sub-region is assigned a second label indicating that there is no pedestrian. In this scenario, the value of the second label can be -1.

[0086] See also Figure 5 Step 2534: Based on the distribution of the first and second class labels in the image region to be tested, determine whether there is a second classification result for pedestrians in the image region to be tested.

[0087] It is understandable that by thresholding h(i) of each sub-region of the image region to be tested, discrete labels can be obtained. The distribution of the first and second type of labels in the image region to be tested can be represented as (t1, t2, ..., t...). 55 ), where t1-t 55 The value is 1 or -1.

[0088] In this embodiment, by converting continuous confidence sub-scores into discrete class labels, the data format for subsequent processing is simplified, and preparation is made for image region-level decision-making based on label distribution.

[0089] In step 2534, the specific implementation of determining whether there are pedestrians in the image region based on the distribution of the first and second type of labels in the image region to be tested can be carried out according to the following steps: The mean-shift segmentation algorithm is used to optimize the initial label distribution of the first and second categories of labels in the image region to be tested, resulting in an optimized label distribution. If the proportion of the image sub-regions labeled with the first category in the optimized label distribution is greater than the preset proportion, then the image region to be tested is determined to be a second classification result indicating the presence of pedestrians.

[0090] In this embodiment, the mean shift weight can be set to |h(Ci)|.

[0091] In this embodiment, the optimized label distribution obtained after optimization can be represented as: .in, The value is 1 or -1. A value of 1 indicates that there are pedestrians in the sub-region of the image, while a value of -1 indicates that there are no pedestrians in the sub-region of the image, or that there are occlusions or background parts.

[0092] In this embodiment, the preset percentage can be set to 80%, that is... If the proportion of values ​​with a median of 1 is above 80%, then the region of the image being tested is identified as a second classification result indicating the presence of pedestrians.

[0093] In this embodiment, by using the mean shift algorithm to optimize the initial label distribution and making decisions based on the proportion of optimized labels, local misjudgment noise is effectively eliminated, and the accuracy and robustness of the overall judgment result are improved in complex situations such as pedestrians being partially occluded.

[0094] See also Figure 2 Step 260: Based on the first classification result or the second classification result corresponding to each image region to be tested, mark the regions in the current frame night view image where pedestrians are determined to exist, and obtain the target night view image.

[0095] It is understandable that after obtaining the classification results of whether there are pedestrians in the area of ​​the image to be tested, it is possible to determine which areas in the current frame of the night view image contain pedestrians. Therefore, by marking the areas where pedestrians are present, the target night view image can be obtained.

[0096] See also Figure 2 Step 270: Display the target night view image in the display unit.

[0097] In some implementations, step 270 may be performed as follows: Figure 6 The following steps are to be performed: See Figure 6 The diagram illustrates a detailed process for displaying a target night view image in a display unit according to an embodiment of this application, specifically including steps 271 to 272: Step 271: Scaling and cropping the target night view image to obtain a new target night view image that is adapted to the display window of the display unit.

[0098] Step 272: Display the new target night view image in the display window.

[0099] In step 271, the target night view image is scaled and cropped, and at least the following three strategies exist: Strategy 1: Full-screen display strategy The aspect ratio of the target night view image often differs from that of the display window of the display unit, requiring stretching and cropping to fit the display window size. Assume the original resolution of the target night view image is W1 wide and H1 high; the display window is W2 wide and H2 high. In this case, the width W1 of the target night view image needs to be stretched proportionally to W2, and then the height is cropped to the display window height H2.

[0100] Strategy 2: Split-screen display strategy Assume the right side of the split-screen display unit is the display window, with a width of 2W2 / 3 and a height of H2. Based on the full-screen cropping scheme, proportional scaling is performed. First, the width is filled, and the width of the night vision display area is 2W2 / 3. That is, the width of the target night vision image is first scaled and adjusted from W1 to 2W2 / 3, and then its height is cropped to 2H2 / 3.

[0101] Strategy 3: Small Window Display Strategy The small window size is relatively small, with the pedestrian target height greater than the width, and the height not less than 6.9mm, with a recommended size of 9.2mm. Based on this, adaptive cropping is performed.

[0102] In some embodiments of this application, the current status information of the display unit can be obtained after the target night view image is determined. If the display unit is currently in navigation mode, the display unit can be split into two parts: one part for navigation display and the other part for target night view image display.

[0103] If the display unit is not currently in navigation mode, the night view image of the target will be displayed in full screen.

[0104] If the display unit is currently on the homepage, the target night view image can be displayed in a small window mode.

[0105] If the display unit is not currently on the homepage, the target night view image can be displayed in a non-small window mode.

[0106] In this embodiment, by adaptively scaling and cropping the final displayed image, it is ensured that key information can be clearly presented at an appropriate ratio regardless of the display mode (full screen, split screen, etc.), thus optimizing the driver's visual experience.

[0107] In some embodiments of this application, the night vision image to be displayed is processed according to the following steps before being displayed: First, the current frame night vision image captured by the night vision camera unit is acquired; second, at least one test image region is determined from the current frame night vision image, each test image region being composed of multiple image sub-regions; third, for each test image region, the image feature vector of the test image region is extracted, and the image feature vector is input into a pre-trained classifier to obtain a first classification result of whether a pedestrian exists in the test image region; if no first classification result is obtained for the test image region, the image sub-feature vector of each image sub-region in the test image region is extracted; fourth, based on the image sub-feature vector of each image sub-region, a second classification result of whether a pedestrian exists in the test image region is determined; fifth, based on the first or second classification result corresponding to each test image region, the region where a pedestrian is determined to exist is marked in the current frame night vision image to obtain the target night vision image; finally, the target night vision image is displayed in the display unit.

[0108] Therefore, the technical solution provided in this application does not rely on a single classifier for pedestrian detection. Instead, it first uses a trained classifier to quickly make a preliminary judgment on the complete image region to be tested, obtaining a first classification result. For cases where the judgment result is unclear or a reliable first classification result indicating the presence or absence of a pedestrian is not obtained, the detection is not terminated directly, and no detection result is output, or a detection result indicating the absence of a pedestrian is output. Instead, a more refined local judgment mechanism is initiated. In this local judgment mechanism, each sub-region of the image region to be tested is analyzed and judged, thereby determining an accurate second classification result for the image region to be tested. Based on this pedestrian detection scheme, the false negative rate and false positive rate of pedestrians in night vision images can be significantly reduced, thereby greatly improving the reliability of pedestrian detection in night vision images and providing drivers with stable and reliable pedestrian warnings.

[0109] Based on the same inventive concept, this application also provides a vehicle equipped with a night vision camera unit and a display unit.

[0110] See Figure 7 The diagram illustrates the structure of a vehicle according to an embodiment of the present application. The vehicle includes one or more memories 704, one or more processors 702, and at least one computer program (computer program instructions) stored on the memory 704 and executable on the processor 702. When the processor 702 executes the computer program, it implements the method described above.

[0111] Among them, Figure 7 In this document, a bus architecture (represented by bus 700) is used. Bus 700 may include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 702 and memory represented by memory 704. Bus 700 may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 705 provides an interface between bus 700 and receiver 701 and transmitter 703. Receiver 701 and transmitter 703 may be the same element, i.e., a transceiver, providing a unit for communicating with various other devices over a transmission medium. Processor 702 is responsible for managing bus 700 and general processing, while memory 704 can be used to store data used by processor 702 during operation.

[0112] The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored as one or more instructions or codes on or transmitted via a computer-readable medium. Other examples and embodiments are within the scope and spirit of this application and the appended claims. For example, due to the nature of software, the functions described above may be implemented using software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Furthermore, the functional units may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0114] The units described as separate components may or may not be physically separate. Similarly, the components of the control device may or may not be physical units; they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0115] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing computer program instructions, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0116] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An image display method, characterized in that, Applied to vehicles equipped with night vision camera units and display units, the method includes: Acquire the current frame night vision image captured by the night vision camera unit; At least one image region to be tested is determined from the current frame night view image, and each image region to be tested is composed of multiple image sub-regions; For each image region to be tested, the image feature vector of that image region is extracted, and the image feature vector is input into a pre-trained classifier to obtain the first classification result of whether there is a pedestrian in the image region to be tested; If the first classification result of the image region to be tested is not obtained, then extract the image sub-feature vector of each image sub-region in the image region to be tested; Based on the image sub-feature vector of each image sub-region, determine whether there is a second classification result for pedestrians in the image region to be tested; Based on the first or second classification result corresponding to each image region to be tested, the regions where pedestrians are determined to exist are marked in the current frame night view image to obtain the target night view image; The target night view image is displayed in the display unit.

2. The method according to claim 1, characterized in that, The vehicle is also equipped with a panoramic camera unit. Before acquiring the current frame night vision image captured by the night vision camera unit, the method further includes: Obtain the current state parameters of the panoramic camera unit and the night vision camera unit, as well as the environmental parameters of the current environment; If the current state parameter and the environment parameter meet the first preset condition, then the step of obtaining the current frame night vision image captured by the night vision camera unit is executed; If the current state parameter and the environment parameter do not meet the first preset condition, the current frame panoramic image captured by the panoramic camera unit is acquired and displayed in the display unit.

3. The method according to claim 1, characterized in that, The extraction of the image feature vector of the region to be tested includes: Extract the image sub-feature vector of each image sub-region in the image region to be tested; The image sub-feature vectors of each image sub-region in the image region to be tested are concatenated according to preset rules to obtain a concatenated vector; The transpose of the spliced ​​vector is used as the image feature vector of the image region to be tested.

4. The method according to claim 1, characterized in that, The step of inputting the image feature vector into a pre-trained classifier to obtain a first classification result indicating whether a pedestrian exists in the region of the image to be tested includes: The image feature vector is input into the classifier, which calculates the classification confidence score of the image region under test using the following decision function; H(x) =ε+w T ×x Where H(x) represents the classification confidence score; ε represents the bias term of the classifier; w T represents the transpose of the weight vector of the classifier; x represents the image feature vector; If the classification confidence score is greater than the first threshold, a first classification result indicating the presence of a pedestrian is obtained; If the classification confidence score is less than the second threshold, then the first classification result indicates that there is no pedestrian, and the second threshold is less than the first threshold. If the classification confidence score is between the first threshold and the second threshold, it is determined that no first classification result has been obtained for the image region to be tested.

5. The method according to claim 4, characterized in that, The second classification result, which determines whether a pedestrian exists in the image region to be tested based on the image sub-feature vector of each image sub-region, includes: For each sub-region of the image to be tested, the weight sub-vector corresponding to the sub-region is extracted from the weight vector of the classifier; Calculate the classification confidence sub-score corresponding to the sub-region of the image according to the following formula; h(i)=ε i +w i T ×C i Where h(i) represents the classification confidence sub-score, ε i This represents the sub-bias term corresponding to the sub-region of the image; w i T C represents the transpose of the weight sub-vector corresponding to the sub-region of the image; i This represents the image sub-feature vector of the sub-region of the image; Based on the classification confidence sub-score corresponding to each image sub-region in the image region to be tested, determine whether there is a second classification result for pedestrians in the image region to be tested.

6. The method according to claim 5, characterized in that, The second classification result, which determines whether a pedestrian exists in the image region under test based on the classification confidence sub-score corresponding to each image sub-region in the image region under test, includes: For each sub-region of the image region to be tested, the classification confidence sub-score corresponding to the sub-region is compared with a preset decision threshold; If the classification confidence score of the image sub-region is greater than the decision threshold, then the image sub-region is assigned a first-class label indicating the presence of a pedestrian. If the classification confidence sub-score of the image sub-region is not greater than the decision threshold, then the image sub-region is assigned a second-class label indicating that no pedestrians exist. Based on the distribution of the first and second class labels in the image region to be tested, determine whether there is a second classification result for pedestrians in the image region to be tested.

7. The method according to claim 6, characterized in that, The second classification result, which determines whether there is a pedestrian in the image region based on the distribution of the first and second type of labels in the image region to be tested, includes: The initial label distribution of the first and second categories of labels in the image region to be tested is optimized using the mean-shift segmentation algorithm to obtain the optimized label distribution. If the proportion of image sub-regions labeled with the first category in the optimized label distribution is greater than the preset proportion, then the image region to be tested is determined to be a second classification result indicating the presence of pedestrians.

8. The method according to claim 1, characterized in that, The method further includes: Obtain a set of color training images; Each color training image in the color training image set is converted to grayscale to obtain a grayscale training image set. Each grayscale training image in the grayscale training image set is subjected to exponential transformation to obtain a sample training image set simulating a night vision environment. Extract the sample feature vector of each sample training image in the sample training image set to obtain the sample feature vector set; Based on the feature vector set, the pre-built support vector machine model is trained to obtain the classifier.

9. The method according to claim 1, characterized in that, Displaying the target night view image in the display unit includes: The target night view image is scaled and cropped to obtain a new target night view image that is adapted to the display window of the display unit; The new target night view image is displayed in the display window.

10. A vehicle, characterized in that, The vehicle is equipped with a night vision camera unit and a display unit, and the vehicle also includes one or more processors and one or more memories, wherein the one or more memories store at least one piece of program code, which is loaded and executed by the one or more processors to implement the method as claimed in any one of claims 1 to 9.

Images