Blind zone early warning method and device based on surround view system, vehicle and storage medium

Abstract

The invention provides a blind area early warning method and device based on a look-around system, a vehicle and a storage medium, and the method comprises the steps: obtaining a current look-around image, and recognizing an object in the current look-around image and the type of the object; obtaining a plurality of continuous historical look-around images, and obtaining the relative movement speed of each object according to the current look-around image and the plurality of continuous historical look-around images; according to the relative movement speed, the object type and the object number, obtaining an object weighted characteristic value, and according to the current operation state information and the current driving environment information of the vehicle, obtaining an operation state weighted characteristic value and a driving environment weighted characteristic value respectively; according to the object weighted feature value, the running state weighted feature value and the driving environment weighted feature value, a trained early warning strategy classification model is adopted, an early warning strategy is obtained, and blind area early warning is completed. Based on multi-dimensional data analysis, it is ensured that a complex traffic environment can be accurately responded in real time, and the adaptability of blind area early warning is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle driving safety technology, and in particular to a blind spot warning method, device, vehicle and storage medium based on a surround view system. Background Art

[0002] As road traffic becomes increasingly busy and the number of vehicles continues to increase, the vehicle blind spot warning system is becoming increasingly important in improving driving safety. Blind spots refer to areas around the vehicle that are difficult for the driver to directly observe from a normal perspective. These areas often hide other vehicles or pedestrians. If these obstacles are not discovered in time, it may lead to more serious traffic accidents. The blind spot warning system provides timely warning information to the driver by monitoring and detecting these areas in real time, thereby reducing the risk of collision caused by blind spots and improving overall driving safety.

[0003] Existing blind spot warnings mainly rely on cameras installed on the front, sides and rear of the vehicle to issue warnings by monitoring objects in the blind spots around the vehicle in real time. Usually, a fixed distance threshold and a preset alarm level are used to issue visual or auditory warnings to the driver based on the type and distance of the detected object. For example, the side rear camera can monitor dynamic objects on both sides and behind the vehicle in real time. When an approaching vehicle or pedestrian is detected, the driver is reminded through the warning light on the side mirror or the sound prompt in the cab.

[0004] However, the existing technology cannot adapt to the changing driving environment and complex traffic conditions by relying only on the information of blind spot objects for early warning and lacking real-time monitoring and analysis of the vehicle's operating status and driving environment. For example, when the vehicle is driving at high speed, making sharp turns or in bad weather conditions, the threat level of obstacles in the blind spot may change rapidly, and the static preset strategy of the existing early warning cannot respond to these changes in time, resulting in excessive or insufficient early warning signals. This affects the reliability and effectiveness of the early warning. Therefore, there is an urgent need for a blind spot early warning method based on a surround view system to solve the technical problem of poor adaptability of blind spot early warning in the existing technology. Summary of the invention

[0005] The present application provides a blind spot warning method, device, vehicle and storage medium based on a surround view system, so as to solve the problem of poor adaptability of blind spot warning in the prior art.

[0006] In a first aspect, the present application provides a blind spot warning method based on a surround view system, which is applied to a vehicle equipped with a surround view system, and the method comprises:

[0007] Obtaining a current surround view image of the vehicle through a preset surround view system, identifying all objects in the current surround view image, and obtaining an object category of each object; wherein the current surround view image includes a blind spot range of the driver's field of vision;

[0008] Acquire a plurality of continuous historical surround view images corresponding to the current surround view image, and acquire a relative moving speed between each object and the vehicle based on all objects in the current surround view image and the current surround view image and the plurality of continuous historical surround view images;

[0009] Obtaining a weighted feature value of the object according to the relative moving speed and the object category of the object and the number of objects contained in the current surround image, and obtaining a weighted feature value of the running state according to the current running state information of the vehicle, and obtaining a weighted feature value of the driving environment according to the current driving environment information of the vehicle;

[0010] According to the weighted feature value of the object, the weighted feature value of the running state and the weighted feature value of the driving environment, a trained warning strategy classification model is used to obtain a warning strategy, and blind spot warning is completed according to the warning strategy.

[0011] In a possible design, the obtaining of the warning strategy by using a trained warning strategy classification model according to the weighted feature value of the object, the weighted feature value of the running state and the weighted feature value of the driving environment includes:

[0012] According to the weighted feature value of the object, the weighted feature value of the running state and the weighted feature value of the driving environment, a trained support vector machine model is used to obtain a warning strategy;

[0013] According to the early warning strategy, the early warning is completed, including:

[0014] If the warning strategy is a first-level warning strategy, a continuous sound alarm is sounded, and a text alarm message is sent to the central control screen of the vehicle, so that the central control screen can present the text alarm message visually in the form of flashing text; or,

[0015] If the warning strategy is a secondary warning strategy, a sound alarm is issued when the time threshold is met continuously, and a text alarm message is sent to the central control screen of the vehicle, so that the central control screen can present the text alarm message visually in the form of flashing text; or,

[0016] If the warning strategy is a level three warning strategy, a text alarm message is sent to the central control screen of the vehicle, so that the central control screen can present the text alarm message visually in the form of permanently lit text.

[0017] In a possible design, the acquiring, based on all objects in the current surround view image and according to the current surround view image and the plurality of continuous historical surround view images, a relative moving speed of each object and the vehicle includes:

[0018] Based on all the objects, a multi-target tracking algorithm is used according to the current surround view image and the plurality of continuous historical surround view images to obtain the trajectory information of each object;

[0019] According to the trajectory information of each object, a Euclidean distance calculation formula is used to obtain the moving distance of each object;

[0020] The relative moving speed of each of the objects is acquired according to the image acquisition time of the current surround view image, the image acquisition time of each of the historical surround view images, and the moving distances of all the objects.

[0021] In a possible design, acquiring a plurality of continuous surround view images corresponding to the current surround view image includes:

[0022] The image acquisition time of the current surround view image is read, and a plurality of continuous historical surround view images corresponding to the image acquisition time are acquired from a database according to a preset time window threshold.

[0023] In a possible design, the method of obtaining the current surround view image of the vehicle through a preset surround view system, identifying all objects in the current surround view image, and obtaining the object category of each object includes:

[0024] Based on multiple image acquisition devices deployed at the front, left, right and rear of the vehicle, multiple image data of the vehicle's surrounding environment are collected and acquired according to a preset sampling frequency;

[0025] According to the multiple image data collected at the same time, an image stitching algorithm is used to obtain a stitching image of the surrounding environment of the vehicle;

[0026] Performing image correction on the stitched image to obtain the current surround image; wherein the image stitching algorithm includes one of feature matching stitching, transformation stitching or a deep learning-based stitching algorithm;

[0027] According to the current surround image, a region growing algorithm is used to segment the current surround image to obtain a plurality of segmented images;

[0028] According to the plurality of segmented images, feature extraction is performed on the segmented images to obtain feature information of each segmented image;

[0029] According to all the feature information, a support vector machine classification algorithm is used to identify the object corresponding to each feature information, and obtain the object category corresponding to the object.

[0030] In a possible design, obtaining the object weighted feature value according to the relative moving speed and the object category of the object and the number of objects contained in the current surround image includes:

[0031] According to the object category of the object, query and obtain the category parameter and the category weight value corresponding to the object category, and calculate and obtain the category weighted feature value through weighted summation based on the category parameters and the category weight values ​​of all the objects;

[0032] According to the relative moving speed of the object, query and obtain a speed weighted value corresponding to the relative moving speed, and calculate and obtain a speed weighted characteristic value by weighted summation based on the relative moving speed and the speed weighted value;

[0033] According to the category weighted feature value, the speed weighted feature value and the number of objects, the object weighted feature value corresponding to the current environment is obtained by preset weight threshold.

[0034] In a possible design, the current driving environment information includes at least one of ground friction coefficient, light intensity and rainfall information;

[0035] Then, according to the current driving environment information of the vehicle, a driving environment weighted characteristic value is obtained, including:

[0036] Obtain at least one of the ground friction coefficient, light intensity and rainfall information of the current environment;

[0037] According to at least one of the ground friction coefficient, light intensity and rainfall information, query and obtain a weighted value corresponding to at least one of the ground friction coefficient, light intensity and rainfall information;

[0038] According to at least one of the ground friction coefficient, light intensity and rainfall information, and a weighted value corresponding to at least one of the ground friction coefficient, light intensity and rainfall information, a weighted characteristic value of the driving environment is calculated by weighted summation.

[0039] In one possible design, the current running state information includes at least one of vehicle speed, steering state and yaw angle;

[0040] Then, according to the current running state information of the vehicle, a weighted characteristic value of the running state is obtained, including:

[0041] According to the current running state of the vehicle, obtaining at least one of the vehicle speed, the steering state and the yaw angle;

[0042] According to at least one of the vehicle speed, the steering state and the yaw angle, query and obtain a weighted value corresponding to at least one of the vehicle speed, the steering state and the yaw angle;

[0043] The weighted characteristic value of the running state is calculated and obtained by weighted summation based on at least one of the vehicle speed, the steering state and the yaw angle, and a weighted value corresponding to at least one of the vehicle speed, the steering state and the yaw angle.

[0044] In one possible design, after identifying all objects in the current surround image and obtaining the object category of each of the objects, the method further includes:

[0045] According to the object category, a color frame matching the object category is used to select the object in the current surround view image to obtain a marked surround view image;

[0046] The marked surround view image is sent to the central control screen of the vehicle so that the central control screen can visualize the marked surround view image.

[0047] In a second aspect, the present application provides an early warning device, comprising:

[0048] An object recognition module, used to obtain a current surround view image of the vehicle through a preset surround view system, identify all objects in the current surround view image, and obtain an object category of each of the objects;

[0049] a speed calculation module, configured to obtain a plurality of continuous historical surround view images corresponding to the current surround view image, and based on all objects in the current surround view image, obtain a relative moving speed between each object and the vehicle according to the current surround view image and the plurality of continuous historical surround view images;

[0050] A weighted feature calculation module, used to obtain a weighted feature value of the object according to the relative moving speed and the object category of the object and the number of objects contained in the current surround image, and to obtain a weighted feature value of the running state according to the current running state information of the vehicle, and to obtain a weighted feature value of the driving environment according to the current driving environment information of the vehicle;

[0051] The warning execution module is used to obtain the warning strategy according to the weighted feature value of the object, the weighted feature value of the operating state and the weighted feature value of the driving environment, using the trained warning strategy classification model, and complete the blind spot warning according to the warning strategy.

[0052] In a third aspect, the present application provides a vehicle, comprising: a processor, and a memory communicatively connected to the processor;

[0053] The memory stores computer-executable instructions;

[0054] The processor executes the computer-executable instructions stored in the memory to implement the early warning method.

[0055] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement an early warning method when executed by a processor.

[0056] The blind spot warning method, device, vehicle and storage medium based on the surround view system provided by the present application can monitor various objects in the surrounding environment of the vehicle in real time by obtaining the current surround view image of the vehicle and identifying the objects therein through the preset surround view system, so as to ensure that the driver has a clear understanding of the obstacles in the blind spot. Through accurate object classification, not only can objects of different categories such as pedestrians, other vehicles or obstacles be identified, but also accurate information can be provided to the driver. The comprehensive coverage of the surround view system enables the driver to obtain a wider range of vision, thereby effectively improving the safety of driving. By obtaining multiple continuous historical surround view images corresponding to the current surround view image, and analyzing the relative moving speed of each object and the vehicle based on the current image and the historical image, the dynamic changes of the object can be monitored in real time. By determining the relative movement trend between the object and the vehicle, the perception of the surrounding environment is further improved. Through the comprehensive analysis of the weighted eigenvalues ​​of the object, the weighted eigenvalues ​​of the running state and the weighted eigenvalues ​​of the driving environment, the degree of danger of each situation can be calculated more accurately, and a flexible warning scheme can be obtained through the trained warning strategy model. This multi-dimensional data analysis based on objects, operating status and environmental characteristics ensures real-time and accurate response to complex traffic environments and timely warning of potential dangerous situations, thereby achieving higher safety in various driving scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 A schematic diagram of a blind spot warning method based on a surround view system provided in an embodiment of the present application;

[0059] Figure 2 A schematic diagram of a method flow for executing an early warning according to an early warning strategy provided in an embodiment of the present application;

[0060] Figure 3A schematic flow chart of a method for obtaining the relative moving speed of each object and the vehicle provided in an embodiment of the present application;

[0061] Figure 4 A schematic flow chart of a method for identifying and obtaining objects and their object categories in a current surround image provided by an embodiment of the present application;

[0062] Figure 5 A schematic diagram of the process flow of the region growing algorithm provided in the embodiment of the present application;

[0063] Figure 6 A schematic diagram of a method flow for obtaining weighted feature values ​​of an object provided in an embodiment of the present application;

[0064] Figure 7 A schematic flow chart of a method for obtaining weighted characteristic values ​​of a driving environment provided in an embodiment of the present application;

[0065] Figure 8 A schematic diagram of a method flow for obtaining weighted characteristic values ​​of operating states provided in an embodiment of the present application;

[0066] Fig. 9 A schematic diagram of a method flow chart for visually presenting a current surround view image provided in an embodiment of the present application;

[0067] Fig.10 A schematic diagram of the structure of the early warning device provided in the embodiment of the present application;

[0068] Fig.11 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims, rather than all embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0070] Existing vehicle surround view systems usually use multiple cameras installed on the front, left and right sides, and rear to generate panoramic surround view images using image stitching algorithms to achieve all-round monitoring of the vehicle's surroundings. Warnings are issued based on the information of objects in the panoramic image. However, warnings based on preset strategies cannot respond to changes in the vehicle's own operating status and operating environment in a timely manner, which can easily lead to excessive or insufficient warning signals.

[0071] Based on the above problems and needs, the inventive concept of the present application is to provide a blind spot warning method, which flexibly determines the warning strategy by comprehensively analyzing the vehicle's operating status, surrounding environment information and the specific conditions of the surrounding objects to improve driving safety. First, the vehicle's current surround view image is obtained, and all objects and their categories in the image are identified. Subsequently, multiple continuous historical surround view images corresponding to the current surround view image are obtained, and the moving speed of each object relative to the vehicle is calculated based on these image data. By calculating the weighted feature values ​​of the relative moving speed, category and number of the object, and combining the vehicle's operating status information and driving environment information, the corresponding weighted feature values ​​are generated respectively. These weighted feature values ​​are used as inputs to the trained warning strategy classification model, and the model accurately classifies the warning strategy that adapts to the current situation according to the input features. Finally, the corresponding blind spot warning operation is performed according to the warning strategy to ensure that the driver can detect and respond to the danger in time. Through the comprehensive processing of multi-dimensional data and flexible decision-making, the present application realizes real-time monitoring and warning of vehicle blind spot risks, and improves driving safety and the driver's risk perception ability.

[0072] In urban traffic environments, when vehicles frequently change lanes, merge lanes, or drive on congested roads, blind spot warnings can effectively identify and alert drivers to hidden pedestrians, non-motor vehicles, and other vehicles around them, thereby improving driving safety. During highway driving, by real-time monitoring of the moving speed and trajectory of dynamic objects around the vehicle, early warning of possible collision risks can be provided to help drivers take risk avoidance measures in a timely manner and reduce the occurrence of traffic accidents. In severe weather conditions, such as rainy days, foggy days, or when driving at night, image processing and multi-source data fusion can still be used to ensure accurate identification and warning of objects in blind spots, thereby enhancing the driver's environmental perception ability. In addition, the present application is also applicable to autonomous driving vehicles, which can achieve more accurate environmental monitoring and decision support through integration with autonomous driving control systems, further improving the safety and reliability of autonomous driving. In short, the warning method of the present application is widely applicable to various driving environments, and can provide timely and accurate safety prompts in different driving scenarios, improve driving experience, and reduce accident risks.

[0073] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0074] Figure 1 This is a flow chart of a blind spot warning method based on a surround view system provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0075] S11, obtaining a current surround view image of the vehicle through a preset surround view system, identifying all objects in the current surround view image, and obtaining an object category of each object; wherein the current surround view image includes a blind spot range of the driver's field of vision.

[0076] In this embodiment, the current surround image covers the environment around the vehicle, especially the blind area of ​​the driver's field of vision. During the image acquisition process, the surround system simultaneously captures the environment around the vehicle from different angles to obtain image data from multiple perspectives. Next, the current surround image is analyzed and processed, and all objects in the image, including pedestrians, vehicles, non-motor vehicles, obstacles, etc., are first identified, and the objects are classified. Object categories include but are not limited to pedestrians, two-wheeled vehicles, four-wheeled vehicles, fixed obstacles, etc., and these object category information will be recorded and classified. The blind area of ​​vision contained in the current surround image is a blind spot that is difficult for the driver to detect and is easy to hide potential dangerous objects. For this reason, through the surround image, it is possible to ensure comprehensive monitoring and accurate classification of all objects in the vehicle's surrounding environment. In particular, the effective identification of potential obstacles in the blind area of ​​vision not only improves the detection coverage and accuracy, but also provides a data basis for subsequent mobile speed estimation and weighted eigenvalue calculation.

[0077] S12, obtaining a plurality of continuous historical surround view images corresponding to the current surround view image, and obtaining a relative moving speed between each object and the vehicle based on all objects in the current surround view image and the current surround view image and the plurality of continuous historical surround view images.

[0078] In this embodiment, by obtaining a plurality of continuous historical surround images corresponding to the current surround image, the motion trajectory of the target object in the time series can be analyzed. Specifically, a plurality of historical surround images corresponding to the current surround image are extracted from the stored historical data. The time interval of the historical images is generally set between several hundred milliseconds and several seconds to ensure sufficient tracking of the object's motion. On this basis, the spatial position change of each object is obtained. By positioning the object in the continuous surround image, the displacement change of the object in the time dimension can be obtained. Then, based on the position change of each object in the adjacent historical surround image, the image acquisition time of the historical surround image of the image of the current surround image is combined to obtain the relative moving speed of the object. In this way, not only can the motion state of the object in the blind spot be analyzed in real time, but also the future dynamics of the object can be predicted, providing a more accurate basis for subsequent blind spot warnings. This embodiment improves the recognition and prediction capabilities of potential dangerous objects in complex dynamic environments by calculating the relative moving speed of the object, thereby achieving a more real-time blind spot warning.

[0079] S13, obtaining the weighted characteristic value of the object according to the relative moving speed and the object category of the object and the number of objects contained in the current surround image, obtaining the weighted characteristic value of the running state according to the current running state information of the vehicle, and obtaining the weighted characteristic value of the driving environment according to the current driving environment information of the vehicle.

[0080] In this embodiment, according to the relative moving speed and object category of each object, the corresponding category parameters and speed weighted values ​​are queried and obtained. The category parameters reflect the importance of different categories of objects in driving safety. For example, the pedestrian category may have a higher weighted value because pedestrians belong to the high-risk category; and non-motor vehicles and motor vehicles set different weighted values ​​according to their potential threat to vehicle driving safety. The speed weighted value is set according to the relative moving speed of the object, and a higher moving speed sets a higher weighted value. Then, according to the number of objects contained in the current surround image, the overall weighted feature value is adjusted. The increase in the number of objects indicates that the complexity of the traffic environment has increased, so the weighted feature value will be adjusted accordingly to reflect the situation in a multi-object environment. The object weighted feature value is obtained by combining the category parameters and speed weighted values ​​of each object and the number of objects. The object weighted feature value can fully reflect the degree of influence of various objects on the vehicle in the current driving environment, and provide basic data for the subsequent early warning strategy formulation.

[0081] Furthermore, the weighted characteristic value of the running state is obtained according to the current running state information of the vehicle. The running state information reflects the dynamic state of the vehicle at the current moment, such as when the vehicle is at high speed and sharp turn, the response time of the vehicle to environmental changes is shortened. The weighted characteristic value of the running state can reflect the impact of the current dynamic state of the vehicle on the warning strategy, ensuring that the warning strategy can be adjusted according to the actual running conditions of the vehicle. At the same time, the weighted characteristic value of the driving environment is obtained according to the current driving environment information of the vehicle. The driving environment information affects the controllability and perception ability of the vehicle. For example, in rainy days or when the road is slippery, the braking distance of the vehicle is extended and the control difficulty increases. At this time, the warning level needs to be increased accordingly. The weighted characteristic value of the driving environment can reflect the impact of the current driving environment on the driving safety of the vehicle, ensuring that the warning system can make appropriate responses under different environmental conditions. In summary, by comprehensively considering the relative moving speed of the object, the object category, the number of objects, the vehicle running state and the driving environment information, the weighted characteristic value of the object, the weighted characteristic value of the running state and the weighted characteristic value of the driving environment are obtained. These data provide comprehensive and accurate data support for the subsequent warning strategy classification model, enabling the most appropriate warning strategy to be output based on the real-time dynamically changing driving environment and vehicle status, thereby effectively improving the adaptability and flexibility of blind spot warning.

[0082] S14, according to the weighted feature value of the object, the weighted feature value of the running state and the weighted feature value of the driving environment, the trained warning strategy classification model is used to obtain the warning strategy, and the blind spot warning is completed according to the warning strategy.

[0083] In this embodiment, according to the weighted feature value of the object, the weighted feature value of the running state and the weighted feature value of the driving environment, these data are input through the trained warning strategy classification model, and the warning strategy adapted to the current situation can be obtained according to the comprehensive analysis of various features. The warning strategy classification model constructs a decision-making mechanism by learning a large amount of historical data in the training stage, and corresponds different feature value combinations to specific warning strategies. Specifically, the acquisition of the warning strategy is based on the current running state of the vehicle, the changes in the surrounding environment and the behavior pattern of the objects in the blind spot. The feature value obtained by weighted calculation is used to classify the current scene, judge whether a warning is needed, and the level of the warning. When the corresponding warning result is obtained through the warning strategy classification model, different response measures will be taken according to different warning strategies. Exemplarily, the response measures may include sound alarms, image prompts or vehicle behavior adjustments, so as to provide timely safety prompts for the driver. It can improve the accuracy of the vehicle's blind spot warning and improve driving safety. In this way, not only the dynamic monitoring of the blind spot is realized, but also the warning strategy can be flexibly adjusted according to the real-time vehicle status and environmental changes to ensure overall efficiency and adaptability.

[0084] This application obtains and identifies all objects and their categories in the current surround view image of the vehicle through a preset surround view system, ensuring all-round perception of the vehicle's surrounding environment. By obtaining multiple continuous historical surround view images corresponding to the current surround view image, and calculating the relative moving speed of each object and the vehicle based on these images, real-time tracking and analysis of the dynamic behavior of the object is achieved, and the early warning is made more accurate and timely by obtaining the motion state of the object. Combining the relative moving speed of the object, the object category and the number of objects in the current surround view image, the object weighted feature value is obtained, and further combined with the current operating state information and driving environment information of the vehicle, the operating state weighted feature value and the driving environment weighted feature value are obtained. Finally, the trained early warning strategy classification model is used to synthesize the above weighted feature values, obtain the early warning strategy, and complete the blind spot warning according to the strategy. Through the comprehensive analysis of multi-dimensional data, the efficiency and reliability of blind spot warning are achieved, the safety of vehicle driving and the risk perception ability of the driver are improved, and it is ensured that in a complex and changeable driving environment, the driver can promptly know and respond to potential blind spot dangers, and enhance the overall driving safety.

[0085] In a specific embodiment, a specific implementation method is provided for obtaining the early warning strategy in the above step S14. Based on the above embodiment, it includes:

[0086] S141, according to the weighted feature value of the object, the weighted feature value of the running state and the weighted feature value of the driving environment, a trained support vector machine model is used to obtain a warning strategy.

[0087] In this embodiment, the weighted eigenvalues ​​of the object, the weighted eigenvalues ​​of the running state and the weighted eigenvalues ​​of the driving environment are passed as input to the trained support vector machine (SVM) model. The support vector machine model is a classification algorithm based on supervised learning, which has been trained with a large amount of historical data to distinguish early warning strategies of different risk levels. The model achieves efficient classification and prediction of the input feature vector by finding the best segmentation hyperplane in the high-dimensional feature space. In the specific operation process, the weighted eigenvalues ​​of the object, the weighted eigenvalues ​​of the running state and the weighted eigenvalues ​​of the driving environment are first normalized to ensure that each feature is compared and calculated under the same dimension. Subsequently, the normalized eigenvalues ​​are arranged in a preset order to form a unified input feature vector. The feature vector is input into the support vector machine model, and the model analyzes the implementation of the current driving state according to the classification boundaries learned during its training process, and outputs the corresponding early warning strategy category. The application of the support vector machine model enables the acquisition of early warning strategies to be highly accurate and real-time. Through the accurate classification of the comprehensive feature vector, the early warning strategy that best suits the current driving environment and vehicle status can be dynamically selected. For example, when the vehicle is traveling at high speed and surrounded by dense obstacles, the model may output a higher level of warning strategy to ensure that the driver can take necessary safety measures in a timely manner. On the contrary, when the vehicle is traveling at a low speed and there are fewer obstacles, a lower level of warning strategy may be selected to avoid unnecessary alarm interference. This embodiment realizes the dynamic selection of warning strategies by introducing a machine learning algorithm (support vector machine model). This not only improves the adaptability of blind spot warnings in complex and changeable driving environments, but also enhances the accuracy and effectiveness of warnings, thereby effectively reducing the risk of collisions caused by blind spots and improving overall driving safety.

[0088] Furthermore, Figure 2 A flowchart of a method for executing an early warning according to an early warning strategy provided in an embodiment of the present application. Figure 2 As shown, including:

[0089] S21, if the warning strategy is a first-level warning strategy, a continuous sound alarm is issued, and a text alarm message is sent to the central control screen of the vehicle, so that the central control screen can visualize the text alarm message in the form of flashing text.

[0090] In this embodiment, when the warning strategy is determined to be a first-level warning strategy, multiple emergency response mechanisms will be triggered to ensure that the driver can quickly perceive and respond. First, a continuous sound alarm will be activated. The sound alarm has a high volume and a sense of urgency, and usually uses a high-frequency and rapid alarm sound to ensure that it can attract the driver's attention in various noise environments. The duration of the sound alarm is usually set according to actual needs, generally a fixed duration or dynamically adjusted according to traffic conditions and environmental changes to ensure that the driver does not miss any important warnings. At the same time, a text alarm message will be sent to the vehicle's central control screen to further enhance the warning effect. The text alarm message usually includes key information related to the current blind spot threat, such as object type, threat level, relative distance and speed of the object, etc., to ensure that the driver can understand the current dangerous situation in a short time. In order to further improve the visibility and sense of urgency of the information, the vehicle's central control screen will use text flashing to present the alarm information. Text flashing can not only effectively attract the driver's attention and reduce neglect caused by visual fatigue or environmental interference, but also strengthen the urgency of the warning information.

[0091] Among them, the frequency and duration of text flashing can be optimized according to actual conditions to avoid visual fatigue or interference caused by flashing too frequently or for too long. For example, in the case of a first-level warning, the flashing frequency may be set to two or three times per second, and the alarm information will continue to be displayed until the driver takes corresponding actions, such as turning the steering wheel, stepping on the brakes, etc., or until the condition of the vehicle changes and the alarm of this level is lifted. This multiple alarm method, that is, the combination of sound and visual alarms, acts simultaneously through two different sensory channels to ensure that the driver can perceive the danger at the first time, enhancing the warning effect and reliability in complex driving environments. In addition, this alarm mechanism also provides drivers with multiple perception channels, ensuring that the warning information is not ignored even in the case of loud noise or limited vision, thereby improving driving safety.

[0092] S22, if the warning strategy is a secondary warning strategy, a sound alarm is issued when the time threshold is continuously met, and a text alarm message is sent to the central control screen of the vehicle, so that the central control screen can visualize the text alarm message in the form of flashing text.

[0093] In this embodiment, when the warning strategy is determined to be a secondary warning strategy, the response mechanism will be adjusted compared to the primary warning strategy, but still maintain a high level of alertness to ensure that the driver can take corresponding actions in more urgent situations. First, the sound alarm will be activated, but compared with the primary warning, the sound alarm of the secondary warning is set to an alarm mode that continuously meets the time threshold. This means that the sound alarm will continue for a period of time after it is activated until the current condition of the vehicle meets the preset release condition or reaches the preset alarm time limit. The frequency and intensity of the alarm sound may be slightly reduced compared to the primary warning, but it is still guaranteed to be enough to attract the driver's attention. The time threshold of the sound alarm is usually set according to different driving environments and danger levels. For example, when the detected potential risk belongs to the secondary threat, the duration of the sound alarm may be set to 5 seconds to 10 seconds, during which the alarm sound is repeated to remind the driver to pay attention. However, if the driver fails to take corresponding actions in time (such as slowing down, changing lanes, etc.) within this time, the alarm volume or frequency can be further increased to ensure that the warning effect is maximized.

[0094] At the same time, the relevant text warning information will be sent to the vehicle's central control screen to further enhance the delivery of warning information. Similar to the first-level warning strategy, the text warning information in the second-level warning strategy also contains key information about the threat, such as the type of object, distance, relative speed, and potential risks associated with the object. The text information is displayed in a flashing manner, which helps drivers more easily capture the warning content during busy driving through dynamic visual effects. However, unlike the first-level warning, the text flashing frequency and duration of the second-level warning are slightly different. In order to avoid visual fatigue or excessive interference, the text flashing frequency of the second-level warning may be set to once per second and last no more than a few seconds. This flashing pattern will be repeated until the vehicle condition changes or the driver takes a safe action. In actual application, the flashing frequency and duration can be dynamically adjusted according to the complexity of the current driving environment to ensure that the driver's attention is maximized while reducing excessive interference. Through this multi-level and multi-channel warning method, the second-level warning strategy not only ensures that the driver can clearly understand the potential risks, but also enhances the flexibility and adaptability of the warning through time thresholds and dynamic adjustments. This mechanism can effectively avoid excessively frequent alarm interference while maintaining sufficient vigilance to provide drivers with more intuitive and timely safety warnings.

[0095] S23, if the warning strategy is a level 3 warning strategy, a text alarm message is sent to the central control screen of the vehicle, so that the central control screen can present the text alarm message visually in the form of a constantly lit text.

[0096] In this embodiment, when the warning strategy is determined to be a level 3 warning strategy, a text alarm message is sent to the central control screen of the vehicle to ensure that the driver can perceive the potential risk in time. Level 3 warnings usually correspond to lighter warning signals. In this case, the choice of using a text-on mode instead of a sound alarm or flashing text is to reduce unnecessary psychological pressure or sudden visual interference, especially in normal driving situations, where too frequent or rapid alarms may cause anxiety or distraction of the driver. Therefore, the text-on display mode can provide a persistent and gentle reminder to enable the driver to be alert under mild risk conditions and be ready to take necessary countermeasures at any time. In actual operation, the text alarm content will clearly indicate the current potential risk, such as: "There is an obstacle behind the right side of the vehicle" or "A slowing vehicle appears in front", and provide necessary prompts in a clear and understandable way. This information will continue to be displayed on the central control screen until the driver responds according to traffic conditions or personal judgment or the system recognizes that the risk has been resolved. In this way, the driver can better understand and respond to changes in the environment around the vehicle without disturbing normal driving due to excessive alarms.

[0097] In a specific embodiment, Figure 3 The following is a flow chart of a method for obtaining the relative moving speed of each object and the vehicle provided in the embodiment of the present application. It is an explanation of the specific implementation of the above step S12. Figure 3 As shown, including:

[0098] S31, based on all objects, using a multi-target tracking algorithm according to the current surround image and multiple continuous historical surround images, to obtain trajectory information of each object;

[0099] S32, according to the trajectory information of each object, using the Euclidean distance calculation formula to obtain the moving distance of each object;

[0100] S33, acquiring the relative moving speed of each object according to the image acquisition time of the current surround view image, the image acquisition time of each historical surround view image, and the moving distance of all objects.

[0101] In this embodiment, the motion trajectory of each object at multiple time points is tracked by a multi-target tracking algorithm, and the relative moving speed of each object is calculated according to the motion trajectory of the object. Generally, the surround view system can collect a large amount of image data, and each frame of the image often contains multiple objects, which may include other vehicles, pedestrians, obstacles, etc. In order to ensure the effective tracking of the movement of all objects, a multi-target tracking algorithm is adopted, which is an algorithm that can handle the movement of multiple targets in continuous frames. Common multi-target tracking methods include Kalman filtering, deep learning sorting algorithms, etc. Through these algorithms, each object can be marked in a continuous image, and the position change of each object at different time points can be recorded. First, by analyzing the image difference and object features between the current surround view image and the historical surround view image, the initial position of each object can be identified, and the position of the object can be continuously tracked in subsequent image frames. These position data are the trajectory information of the object, which describes the change of the position of the object in the image over time. Through these trajectory information, the motion mode of each object can be obtained, and the relative speed of the object can be further calculated. Next, the Euclidean distance calculation formula is used to calculate the moving distance of the object between two time points according to the position change of each object in the continuous image. Subsequently, the relative moving speed of each object is calculated by combining the trajectory information of each object, the image acquisition time of the current surround view image, and the image acquisition time of each historical surround view image. Since the image acquisition has a time interval, the speed of the object is calculated according to the time difference (i.e., the acquisition time of different image frames). In this way, the speed of each object relative to the vehicle can be calculated in real time, and the dynamic characteristics of each object can be obtained. Through dynamic tracking and speed calculation, the motion trajectory of objects around the vehicle can be captured, and then the relative speed between the object and the vehicle can be analyzed. Through this process, more accurate basic data is provided for subsequent blind spot warnings. This ensures the real-time and accuracy of blind spot monitoring while improving the driving safety of the vehicle.

[0102] Further, based on the above embodiment, obtaining a plurality of continuous surround view images corresponding to the current surround view image includes:

[0103] S121, reading the image acquisition time of the current surround view image, and acquiring a plurality of continuous historical surround view images corresponding to the image acquisition time from a database according to a preset time window threshold.

[0104] In this embodiment, the image acquisition time is the timestamp of each image captured and recorded in the surround view system. Each time an image is acquired, the surround view system will add a timestamp (usually accurate to milliseconds) to the image to identify the shooting time of the image. In order to obtain historical data related to the current surround view image, a preset time window threshold needs to be set. This time window defines the time range for retrospectively acquiring historical images. For example, if the preset time window threshold is 2 seconds, all surround view images within 2 seconds from the timestamp of the current surround view image will be retrospectively retrieved. This time window threshold is set based on the actual driving speed of the vehicle, road conditions and the real-time processing capability of the system, in order to ensure that the historical image data is sufficient to cover the moving trajectory of the object and has sufficient time resolution when calculating the object speed and trajectory. By presetting the time window threshold, multiple continuous historical surround view images corresponding to the current timestamp can be queried from the database. These historical images contain the surrounding environment information of the vehicle in the past period of time. Specifically, a query request will be sent to the database based on the timestamp of the current surround view image to retrieve historical images that meet the time requirements. For example, assuming that the timestamp of the current surround view image is T1 and the preset time window is 2 seconds, image data with timestamps between T1-2 seconds and T1 will be retrieved from the database. These historical image data provide a data basis for the dynamic analysis of objects. After obtaining these historical surround view images, by comparing the differences between the current image and the historical images, the relative position changes and movement trajectories of the objects can be analyzed. On this basis, combined with the time information, the dynamic characteristics of the object, such as the relative speed and acceleration, can be calculated. It can be seen that the acquisition of historical surround view images not only improves the responsiveness to dynamic environments, but also effectively enhances the vehicle's detection and warning capabilities for blind spot objects, ensuring the driving safety of vehicles in complex traffic scenarios.

[0105] In one embodiment, Figure 4 The following is a flow chart of a method for identifying and obtaining objects and their object categories in the current surround image provided by the embodiment of the present application. It is an explanation of the specific implementation of the above step S11. Based on the above embodiment, Figure 4 As shown, including:

[0106] S41, based on multiple image acquisition devices deployed at the front, left, right and rear of the vehicle, collecting and acquiring multiple image data of the vehicle surrounding environment according to a preset sampling frequency;

[0107] S42, using an image stitching algorithm to obtain a stitched image of the vehicle's surrounding environment based on multiple image data collected at the same time;

[0108] S43, performing image correction on the stitched image to obtain a current surround image; wherein the image stitching algorithm includes one of feature matching stitching, transformation stitching or a deep learning-based stitching algorithm;

[0109] S44, segmenting the current surround view image using a region growing algorithm to obtain a plurality of segmented images;

[0110] S45, performing feature extraction on the segmented images according to the multiple segmented images to obtain feature information of each segmented image;

[0111] S46, based on all the feature information, using a support vector machine classification algorithm to identify the object corresponding to each feature information, and obtaining the object category corresponding to the object.

[0112] In this embodiment, multiple image acquisition devices (such as cameras) deployed around the vehicle collect image data of the vehicle's surrounding environment in real time according to a preset sampling frequency. These image acquisition devices are usually distributed in the front, left, right and rear of the vehicle to ensure that the full range of vision around the vehicle is covered. In order to ensure the timeliness and accuracy of image acquisition. Among them, these cameras may use different types of sensor technology, including wide-angle lenses, fisheye lenses or stereo camera systems, to ensure that a complete image of the surrounding environment is captured from different angles. Each image acquisition device will generate a series of image data with timestamps. After collecting image data from multiple different angles at the same time, these images need to be spliced ​​to generate a panoramic surround image. This process uses an image stitching algorithm, which usually includes feature matching stitching, transformation stitching and a stitching algorithm based on deep learning. Among them, feature matching stitching extracts feature points (such as corners, edges, etc.) in the image for matching, and performs geometric transformation and image fusion according to the matching results, so as to stitch images from multiple perspectives into a complete surround image. The feature matching stitching algorithm can effectively process images with large differences in perspectives and ensure that the stitched images have no obvious stitching marks. Transform stitching combines images from multiple perspectives into a smoother panoramic image by performing geometric transformations (such as affine transformation and perspective transformation) on the images. Transform stitching is usually used to process images with small differences in shooting angles, and can quickly generate more accurate surround images. The deep learning-based stitching algorithm can automatically learn the geometric relationships and features between images by training deep neural networks, and then seamlessly stitch images from different perspectives. The deep learning-based stitching method can better handle image fusion problems in dynamic environments and complex scenes.

[0113] In some cases, the stitched surround view image usually still has some distortion (such as perspective distortion or light influence), so the image needs to be corrected. The image correction process mainly eliminates the distortion generated in the stitching process through geometric correction and color correction to ensure the accuracy and consistency of the image. Geometric correction usually includes using camera calibration parameters to eliminate the radial and tangential distortion of the image, so that the shape and position of each object in the image are more accurate. Color correction adjusts the image's hue, brightness, and contrast parameters to make the colors in the stitched image more natural and balanced. After the corrected surround view image is processed, the region growing algorithm is used to segment the image. The region growing algorithm is an image segmentation technology based on pixel similarity. Starting from a certain initial seed pixel, the surrounding similar pixels are gradually included in the region until certain segmentation conditions are met. The algorithm is usually used to identify different regions or objects in the image, such as lanes, pedestrians, stationary objects, etc. In the surround view system, the region growing algorithm can effectively distinguish different objects or regions in the image, providing a basis for subsequent object recognition.

[0114] After image segmentation, feature extraction is performed on multiple segmented areas. Feature extraction is to extract key data that can describe the characteristics of objects from each segmented image. These features usually include but are not limited to color histogram, texture features, shape features and edge features. By extracting these features, objects of different categories, such as pedestrians, vehicles, obstacles, etc., can be effectively distinguished. Finally, based on the extracted features, the support vector machine classification algorithm is used to classify objects in each segmented image. For example, vehicles can be distinguished from other objects such as pedestrians and obstacles by their specific shape and edge features. In this way, each object in the current surround view image can be identified and labeled and classified into specific object categories (such as pedestrians, stationary objects, dynamic objects, etc.). In summary, through the comprehensive acquisition and processing of image data, not only can accurate images of the vehicle's surrounding environment be obtained, but also the high quality and high precision of the surround view image can be ensured through effective image stitching, correction, segmentation and object recognition. These surround view images and recognition results provide a solid foundation for subsequent object detection, tracking, mobile speed calculation and early warning strategy formulation.

[0115] For example, Figure 5 The following is a flow chart of the region growing algorithm provided in the embodiment of the present application. Here, the region growing algorithm is described. Based on the above embodiment, Figure 5As shown, it includes: obtaining the target area image as the image to be processed and preprocessing it. Then, the image is divided into blocks according to rows and columns. In the case of the current row n1, all the pixels from n1-M to n1+M rows are taken as a block. Similarly, for the current column n2, the corresponding block contains all the pixels from n2-M to n2+M columns. In this way, the block division of the entire image is completed. After the block division is completed, the first 10% of the pixels with the largest grayscale value and the last 10% of the pixels with the smallest grayscale value in each block are eliminated, so as to remove the influence of abnormal values ​​on the calculation results. Then, the grayscale value variance of each block is calculated to form the corresponding row grayscale value variance array and column grayscale value variance array. Through the preset first threshold, the blocks with grayscale value variance greater than the threshold are screened out and marked as seed points. After determining the seed point, with the seed point as the core, according to the preset second threshold, the blocks with grayscale value difference less than the threshold are selected and classified as the growth area of ​​the seed point. For these newly added area blocks, repeat the above process to continue to determine whether the growth conditions are met. The seed point growth process will continue until all seed points no longer meet the growth conditions. After the seed point growth is completed, the segmentation images based on rows and columns are obtained respectively. Finally, the row segmentation image and the column segmentation image are taken as the union to form the final segmentation image.

[0116] In a specific embodiment, Figure 6 The flowchart of the method for obtaining the weighted feature value of an object provided in the embodiment of the present application is a specific implementation method of obtaining the weighted feature value of an object in the above step S13. Figure 6 As shown, including:

[0117] S61, according to the object category of the object, query and obtain the category parameter and category weight value corresponding to the object category, and calculate and obtain the category weighted feature value by weighted summation based on the category parameters and category weight values ​​of all objects;

[0118] S62, according to the relative moving speed of the object, query and obtain the speed weighted value corresponding to the relative moving speed, and calculate and obtain the speed weighted characteristic value by weighted summation based on the relative moving speed and the speed weighted value;

[0119] S63, obtaining the object weighted feature value corresponding to the current environment according to the category weighted feature value, the speed weighted feature value and the number of objects through a preset weight threshold.

[0120] In this embodiment, based on the category information of each object identified, the category parameters and category weights corresponding to the object category are queried and obtained. The category parameters quantify the objects to facilitate subsequent calculations, and each category of the object can be matched with a weight. The setting of the object category weights takes into account the different degrees of impact that the objects may have on vehicle safety during driving. After obtaining the category weighted value of each object, a weighted sum is performed based on the category information of all objects in the current surround image. This process combines the category weighted value of each object with the category parameter of the object to obtain a category weighted feature value. For example, in the blind spot of the vehicle, if there are multiple pedestrians, the sum of the weighted values ​​of these pedestrian categories will be calculated to obtain a category weighted feature value that reflects the risk around the current vehicle.

[0121] According to the relative moving speed of each object, query and obtain the speed weighted value corresponding to the speed. The weighted value of the relative moving speed is usually set based on the speed of the object relative to the vehicle. For objects approaching the vehicle faster, a higher weighted value will be set, indicating that the object poses a greater threat to the vehicle. For slower objects, a lower weighted value may correspond, indicating that the threat is less. In this way, a speed weighted value can be assigned to each object according to the moving speed of the object. Then, a weighted sum is performed based on the speed information of all objects to calculate an overall speed weighted eigenvalue.

[0122] After obtaining the weighted eigenvalues ​​of the object category and the weighted eigenvalues ​​of the object speed, this information is combined with the number of objects, and the final weighted summation is performed through the preset weight threshold to obtain the weighted eigenvalues ​​of the objects corresponding to the current environment. The setting of the weight threshold is usually determined based on factors such as the current driving environment of the vehicle, road conditions, and traffic rules. For example, in urban roads, where the number of objects is large and may be dense, a higher weight threshold may be set to emphasize the weighted calculation of object categories and speeds. In summary, by weighted summation of the category weighted eigenvalues, speed weighted eigenvalues, and the number of objects, a comprehensive object weighted eigenvalue can be obtained. This eigenvalue can reflect the comprehensive performance of all objects to the vehicle in the current environment and has strong reliability. It also improves the response speed and accuracy of the blind spot warning system, ensuring that the driver can obtain warning information in time when facing potential dangers, thereby enhancing the safety of the vehicle.

[0123] For example, the quantization assignment is performed according to the object category S: if it is a pedestrian, then S 1 is 3, and the corresponding category weight is θ 1 is 1.5; if it is a non-motor vehicle, then S 2 is 2, and the corresponding category weight is θ 1 =1.25; if it is a motor vehicle, then S3 is 1, and the corresponding category weight is θ 1 =0.75. Using the formula:

[0124] T s ′ =∑S i ×θ 1

[0125] Calculate and obtain the category weighted feature value T s ′ .

[0126] The settings according to the object speed V include: when the object speed is less than 30km / h, θ 2 is 0.75; when the object speed is between 30km / h and 60km / h, θ 2 is 1.25; when the object speed is greater than 60km / h, θ 2 is 1.5, using the formula:

[0127] T v ′ =∑V i ×θ 2

[0128] Calculate the velocity weighted eigenvalue T v ′ .

[0129] The weighted value of the number of objects N (θ 3 ) settings include: when the number of objects is less than or equal to 1, θ 3 is 1; when the number of objects is between 1 and 5, θ 3 is 2; when the number of objects is greater than 5, θ 3 is 3.

[0130] Weight the feature value T according to the category s ′ , velocity weighted eigenvalue T v ′ and the number of objects N, by presetting the weight threshold, using the formula:

[0131] T 1 =T s ′ ×θ 1 ′ +T v ′ ×θ 2 ′ +N×θ 3

[0132] Calculate and obtain the weighted eigenvalue T of the object corresponding to the current environment 1 .

[0133] In one embodiment, the current driving environment information includes at least one of ground friction coefficient, light intensity and rainfall information.

[0134] In this embodiment, the ground friction coefficient, light intensity and rainfall information, these environmental parameters have a certain impact on the vehicle's control ability, early warning strategy and the driver's reaction speed. Specifically, the ground friction coefficient is a parameter that measures the friction between the vehicle tires and the road surface, usually expressed as a numerical value. A higher friction coefficient means that there is stronger traction between the vehicle and the road surface, and the vehicle's handling performance is better, especially more stable during acceleration, braking and steering. On the contrary, a lower friction coefficient means that the road surface is slippery and the vehicle is prone to skidding or losing control, especially in wet or icy environments. In practical applications, the measurement of the ground friction coefficient usually relies on sensors on the vehicle (such as wheel speed sensors, accelerometers, etc.), as well as real-time perception data of the road surface. By monitoring the road surface conditions in real time, an accurate friction coefficient value can be obtained and used as part of the driving environment information in the calculation.

[0135] Light intensity affects the driver's visual perception and the working effect of vehicle sensors. Under low light conditions, the driver's visual range and ability to perceive the surrounding environment will be limited, especially at night, in tunnels or other low-light areas, the driver's ability to identify blind spots, road signs, pedestrians and other traffic participants is poor. Through the on-board ambient light sensor (for example, optical sensor, infrared sensor, etc.), the surrounding light intensity information can be obtained in real time. If the light intensity is lower than the preset threshold, it may mean that the driver's field of vision is limited, and it is necessary to appropriately adjust the warning strategy or issue a warning message. Rainfall is one of the factors that affect the slipperiness of the road and the traction of the vehicle. Excessive rainfall may cause water accumulation and slippery roads, increase the risk of vehicle skidding, and affect the braking and steering stability of the vehicle. Rainfall information is usually obtained by on-board rain sensors (such as wiper sensors) or meteorological monitoring systems. The vehicle can monitor the rainfall intensity in real time and issue warnings in advance when encountering heavy rain to remind the driver to pay attention to driving safety.

[0136] After the vehicle obtains at least one of the ground friction coefficient, light intensity and rainfall information, the weighted characteristic value of the driving environment will be calculated based on this information. This characteristic value can reflect the impact of current road and weather conditions on driving safety, and thus affect the overall response of the blind spot warning system. For example, on a slippery road, the ground friction coefficient is low, which may lead to an increase in braking distance and an increased risk of vehicle loss of control. In a low-light environment, the driver's ability to perceive the surrounding environment is weakened, and objects in the blind spot are more likely to be ignored. On rainy days, slippery roads and limited visibility may increase the probability of traffic accidents. Therefore, an overall weighted characteristic value of the driving environment is calculated based on the weighted values ​​of these environmental information. It can be seen that by real-time monitoring and combining a variety of driving environment information, the triggering conditions of the blind spot warning can be dynamically adjusted. The flexibility and adaptability of the warning are enhanced, so that it can adapt to different driving environment conditions, optimize the driving experience and improve the safety of the vehicle.

[0137] Figure 7 The following is a flow chart of a method for obtaining weighted characteristic values ​​of a driving environment provided in an embodiment of the present application. It is a specific description of obtaining weighted characteristic values ​​of a driving environment in step S13. Figure 7 As shown, including:

[0138] S71, obtaining at least one of ground friction coefficient, light intensity and rainfall information of the current environment;

[0139] S72, querying and obtaining a weighted value corresponding to at least one of the ground friction coefficient, the light intensity and the rainfall information according to at least one of the ground friction coefficient, the light intensity and the rainfall information;

[0140] S73, according to at least one of the ground friction coefficient, light intensity and rainfall information, and a weighted value corresponding to at least one of the ground friction coefficient, light intensity and rainfall information, calculate and obtain a weighted characteristic value of the driving environment through weighted summation.

[0141] In this embodiment, at least one of the ground friction coefficient, light intensity and rainfall is obtained, and the preset weighted values ​​are queried according to these parameters. These weighted values ​​reflect the degree of influence of different environmental conditions on vehicle stability and safety. The ground friction coefficient has a high correlation with the driving stability of the vehicle. A lower friction coefficient usually indicates that the road condition is poor (such as wet, icy), and at this time, a larger weighted value is set for the low friction coefficient. This means that in the case of a low friction coefficient, a blind spot warning may be issued in advance. Similarly, the light intensity is directly related to the driver's field of vision clarity. Therefore, a higher weighted value is assigned to the low light environment. When the precipitation is large, the road surface is prone to water accumulation, affecting the vehicle's handling stability. According to the amount of precipitation, the corresponding weighted value is searched. The greater the precipitation, the higher the weighted value, thereby increasing the sensitivity to the current driving environment. Through the obtained environmental information and its corresponding weighted value, a weighted sum is performed to obtain a comprehensive driving environment weighted characteristic value. This value will serve as the basis for the impact of the vehicle driving environment on the behavior of the warning system. In summary, by real-time monitoring of the ground friction coefficient, light intensity and rainfall information, querying the weighted values ​​and weighted summing them, the triggering conditions of the blind spot warning can be dynamically adjusted according to different driving environments. In more complex or dangerous driving environments (such as heavy rain, low light, etc.), the driver's vigilance will be increased, and warnings will be issued in time to remind the driver of potential dangers in the blind spot. In good driving environments (such as dry and sunny weather and good lighting conditions), the alarm frequency can be reduced to avoid excessive interference with the driver.

[0142] Exemplarily, the vehicle's driving environment information includes the ground adhesion coefficient μ, the light intensity G, and the rainfall information R. When the ground adhesion coefficient is less than 0.8, g 1 When the ground adhesion coefficient is between 0.8 and 1.25, g 1 When the ground adhesion coefficient is between 1.25 and 1.75, g 1 When the ground adhesion coefficient is greater than 1.75, g 1 is 0.5. Light intensity weighted value (g 3 ) is quantified according to different light intensities: when the light intensity is less than 50 lux, g 2 is 1; when the light intensity is between 50lux and 500lux, g 2 is 0.75; when the light intensity is greater than 500 lux, g 2 is 1. Rainfall information weighted value (g 3 ) is set according to the amount of precipitation: when the rainfall is less than 25 mm / day, g 3 is 0.5; when the rainfall is between 25 mm / day and 50 mm / day, g 3 is 0.75; when the rainfall is greater than 50 mm / day, g3 is 1.

[0143] Using the formula:

[0144] T 2 =μ×g 1 +G×g 2 +R×g 3

[0145] Calculate and obtain the weighted characteristic value T of the driving environment 2 .

[0146] In one embodiment, the current operating state information includes at least one of vehicle speed, steering state, and yaw angle.

[0147] In this embodiment, the vehicle speed is a basic parameter describing the vehicle's driving state. The change in vehicle speed directly affects the vehicle's maneuverability, braking distance and stability. Generally speaking, when the vehicle speed is faster, the braking distance becomes longer, especially in complex environments (such as curves, slippery roads, etc.), which may increase the risk of vehicle loss of control. Therefore, the vehicle speed is an important indicator in the current running state information. The vehicle speed data is usually obtained in real time by the vehicle's vehicle speed sensor, wheel speed sensor or GPS positioning system. By monitoring the vehicle speed in real time, the early warning strategy can be dynamically adjusted, such as increasing the monitoring and warning of surrounding objects at high speeds. The steering state describes the vehicle's current steering angle or steering operation. For example, the steering state can reflect whether the vehicle is turning, accelerating the turn or keeping straight, etc. This state is related to the vehicle's trajectory control, especially when driving at high speeds, the steering angle and steering rate have an impact on the vehicle's stability. The steering state data can be obtained through the vehicle's steering system sensors, which can usually detect the steering wheel's rotation angle and rotation rate. By real-time monitoring of the steering state, it can be determined whether the vehicle is making a sharp turn, etc., and the early warning strategy can be optimized accordingly. The yaw angle describes the angle at which the vehicle rotates around an axis perpendicular to the ground. The yaw angle reflects whether the vehicle deviates from the normal driving trajectory during driving. When driving at high speed, if the yaw angle of the vehicle is too large, it may mean that the vehicle is skidding or out of control. Therefore, further real-time monitoring of the vehicle's yaw angle can ensure driving safety. The yaw angle data is usually obtained by the gyroscope or inertial measurement unit (IMU) sensor on the vehicle. Through these sensors, the instantaneous rotation angle of the vehicle and its rate of change are measured.

[0148] Figure 8 The following is a flow chart of a method for obtaining weighted characteristic values ​​of operating status provided in an embodiment of the present application. It is a specific description of obtaining weighted characteristic values ​​of operating status in step S13. Based on the above embodiment, Figure 8 As shown, including:

[0149] S81, acquiring at least one of a vehicle speed, a steering state, and a yaw angle according to a current running state of the vehicle;

[0150] S82, according to at least one of the vehicle speed, the steering state and the yaw angle, query and obtain a weighted value corresponding to at least one of the vehicle speed, the steering state and the yaw angle;

[0151] S83, according to at least one of the vehicle speed, the steering state and the yaw angle, and the weighted value corresponding to at least one of the vehicle speed, the steering state and the yaw angle, calculate and obtain the weighted characteristic value of the operating state through weighted summation.

[0152] In this embodiment, one of the information of vehicle speed, steering state or yaw angle is obtained, and the corresponding weighted value is queried based on these data. The weighted value of vehicle speed is usually related to the size of the vehicle speed itself. When driving at high speed, the braking distance and risk of loss of control of the vehicle are greater, so the weighted value of vehicle speed is higher. The weighted value of steering state is related to the steering angle and steering rate of the vehicle. When making a sharp turn or changing lanes quickly, the weighted value of steering state is larger. The weighted value of yaw angle is directly related to the stability of the vehicle. The weighted value corresponding to a larger yaw angle will also be larger. After obtaining the vehicle speed, steering state or yaw angle and their respective weighted values, the weighted characteristic value of the vehicle's operating state is calculated by weighted summation. The weighted value of each state reflects the degree of influence of the state on the safety of the vehicle. According to different weight settings, the information of vehicle speed, steering state and yaw angle is comprehensively processed to obtain a comprehensive weighted characteristic value of the operating state. The weighted characteristic value of the operating state is a quantitative indicator of the current operating state of the vehicle, so as to adjust the safety strategy according to the different operating states and ensure that the driver can obtain timely and effective safety prompts under complex road conditions.

[0153] Exemplarily, the current running state information includes vehicle speed v, steering state δ and yaw angle β,

[0154] Speed ​​weighted value (α 1 ) is set as follows: When the vehicle speed is less than 30 km / h, α 1 is 0.5; when the vehicle speed is between 30km / h and 60km / h, α 1 is 0.75; when the vehicle speed is greater than 60km / h, α 1 is 1. Steering state weighted value (α 2 ) is determined according to the state of the turn signal: when the turn signal is not turned on, α 2 is 0.5; when the turn signal is left, α 2 is 0.75; when the turn signal is right, α 2 1. Yaw angle weighted value (α 3 ) is assigned according to the size of the yaw angle: when the yaw angle is less than 20°, α 3is 0.5; when the yaw angle is between 20° and 40°, α 3 is 0.75; when the yaw angle is greater than 40°, α 3 is 1.

[0155] Using the formula:

[0156] T 3 =v×α 1 +δ×α 2 +β×α 3

[0157] Calculate and obtain the weighted eigenvalue T of the operating state 3 .

[0158] Then, each weighted eigenvalue T1, T2 and T3 will be used as the input of the support vector machine algorithm. In the training stage, SVM finally realizes the classification of the alarm level by continuously adjusting the parameters of the hyperplane. Specifically, the SVM model optimizes the classification result by selecting different kernel functions (such as linear kernel function, radial basis kernel function (RBF), polynomial kernel function or Sigmoid kernel function) and penalty parameter C. Commonly used kernel functions include linear kernel function, radial basis function kernel, and polynomial kernel function, which are respectively applicable to different types of feature space mapping, and can adjust the complexity and classification accuracy of the model according to actual needs. Finally, SVM takes eigenvalues ​​T1, T2 and T3 as input and outputs an alarm strategy adapted to the current situation. Through the above embodiment, it is possible to calculate the weighted eigenvalues ​​in real time and generate an alarm strategy according to the vehicle state, the surrounding environment and the situation of obstacles, so as to ensure that early warning information can be provided to the driver in different driving environments and improve driving safety.

[0159] Fig. 9 A schematic diagram of a method for visualizing the current surround image provided in an embodiment of the present application. After obtaining the current surround image, and based on the above embodiment, as Fig. 9 As shown, including:

[0160] S91, according to the object category, using a color frame matching the object category to select the object in the current surround view image to obtain a marked surround view image;

[0161] S92, sending the marked surround view image to the central control screen of the vehicle, so that the central control screen can visualize the marked surround view image.

[0162] In this embodiment, according to the category of the object, a color frame matching the object category will be selected. Common object categories include pedestrians, other vehicles, obstacles, etc., and each object category will be set with a specific identification color. For example, pedestrians may be marked with a red frame and other vehicles with a blue frame. This classification and color frame matching method allows the driver to quickly identify different types of potential dangers or obstacles and improve reaction speed. The object selection process is usually completed by a region detection algorithm, which identifies the specific location of the object in the surround view image and draws a corresponding color frame on the boundary of the object. Once the category of the object and the corresponding color frame are determined, these frames will be applied to the identified object in the current surround view image to generate a marked surround view image. This marked surround view image can not only provide the location and category information of the object, but also distinguish it through frames of different colors, further enhancing the visual recognition effect of the object and helping the driver to quickly identify potential risks in the surrounding environment. Subsequently, the marked surround view image is sent to the vehicle's central control screen through the vehicle's communication system. The central control screen, as the main information display interface in the vehicle, can display image information in real time. Through such visual presentation, the driver can quickly obtain complete information about the vehicle's surroundings, which helps improve driving safety and efficiency, especially in operations such as reversing and changing lanes.

[0163] The embodiment of the present invention can divide the functional modules of the vehicle or the main control device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0164] Fig.10 This is a schematic diagram of the structure of the early warning device provided in the embodiment of the present application. Fig.10 As shown, the early warning device 10 includes:

[0165] The object recognition module 101 is used to obtain the current surround view image of the vehicle through a preset surround view system, identify all objects in the current surround view image, and obtain the object category of each object;

[0166] The speed calculation module 102 is used to obtain a plurality of continuous historical surround view images corresponding to the current surround view image, and based on all objects in the current surround view image, obtain a relative moving speed of each object and the vehicle according to the current surround view image and the plurality of continuous historical surround view images;

[0167] The weighted feature calculation module 103 is used to obtain the weighted feature value of the object according to the relative moving speed and the object category of the object and the number of objects contained in the current surround image, and to obtain the weighted feature value of the running state according to the current running state information of the vehicle, and to obtain the weighted feature value of the driving environment according to the current driving environment information of the vehicle;

[0168] The warning execution module 104 is used to obtain the warning strategy according to the weighted feature value of the object, the weighted feature value of the running state and the weighted feature value of the driving environment, using the trained warning strategy classification model, and complete the blind spot warning according to the warning strategy.

[0169] The early warning device 10 provided in this embodiment can execute the early warning method of the above-mentioned embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0170] In the aforementioned specific implementation, each module may be implemented as a processor, and the processor may execute computer-executable instructions stored in the memory, so that the processor executes the above method.

[0171] Fig.11 This is a schematic diagram of the structure of a vehicle provided in an embodiment of the present application. Fig.11 As shown, the vehicle 11 includes: at least one processor 111 and a memory 112. The vehicle 11 also includes a communication component 113. The processor 111, the memory 112 and the communication component 113 are connected via a bus 114.

[0172] In a specific implementation process, at least one processor 111 executes computer-executable instructions stored in the memory 112 , so that at least one processor 111 executes the method executed by the vehicle side as described above.

[0173] The specific implementation process of the processor 111 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0174] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0175] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk storage.

[0176] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0177] The above-mentioned functions implemented by the vehicle and the main control device introduce the scheme provided by the embodiment of the present invention. It can be understood that in order to implement the above-mentioned functions, the vehicle or the main control device includes a hardware structure and / or software module corresponding to the execution of each function. In combination with the units and algorithm steps of each example described in the embodiment disclosed in the embodiment of the present invention, the embodiment of the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present invention.

[0178] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0179] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0180] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a vehicle or a main control device.

[0181] The present application also provides a computer program product, which includes: a computer program, which is stored in a readable storage medium, and at least one processor of a vehicle can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the vehicle executes the solution provided by any of the above embodiments.

[0182] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0183] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A blind spot warning method based on a surround view system, applied to a vehicle equipped with a surround view system, characterized in that: The method comprises: Obtaining a current surround view image of the vehicle through a preset surround view system, identifying all objects in the current surround view image, and obtaining an object category of each object; wherein the current surround view image includes a blind spot range of the driver's field of vision; Acquire a plurality of continuous historical surround view images corresponding to the current surround view image, and acquire a relative moving speed between each object and the vehicle based on all objects in the current surround view image and the current surround view image and the plurality of continuous historical surround view images; Obtaining a weighted feature value of the object according to the relative moving speed and the object category of the object and the number of objects contained in the current surround image, and obtaining a weighted feature value of the running state according to the current running state information of the vehicle, and obtaining a weighted feature value of the driving environment according to the current driving environment information of the vehicle; According to the weighted feature value of the object, the weighted feature value of the running state and the weighted feature value of the driving environment, a trained warning strategy classification model is used to obtain a warning strategy, and blind spot warning is completed according to the warning strategy.

2. The method according to claim 1, characterized in that The step of obtaining the early warning strategy by using the trained early warning strategy classification model according to the weighted feature value of the object, the weighted feature value of the running state and the weighted feature value of the driving environment comprises: According to the weighted feature value of the object, the weighted feature value of the running state and the weighted feature value of the driving environment, a trained support vector machine model is used to obtain a warning strategy; According to the early warning strategy, the early warning is completed, including: If the warning strategy is a first-level warning strategy, a continuous sound alarm is sounded, and a text alarm message is sent to the central control screen of the vehicle, so that the central control screen can present the text alarm message visually in the form of flashing text; or, If the warning strategy is a secondary warning strategy, a sound alarm is issued when the time threshold is met continuously, and a text alarm message is sent to the central control screen of the vehicle, so that the central control screen can present the text alarm message visually in the form of flashing text; or, If the warning strategy is a level three warning strategy, a text alarm message is sent to the central control screen of the vehicle, so that the central control screen can present the text alarm message visually in the form of permanently lit text.

3. The method according to claim 1, characterized in that The acquiring the relative moving speed between each object and the vehicle based on all objects in the current surround view image and the multiple continuous historical surround view images includes: Based on all the objects, a multi-target tracking algorithm is used according to the current surround view image and the plurality of continuous historical surround view images to obtain the trajectory information of each object; According to the trajectory information of each object, a Euclidean distance calculation formula is used to obtain the moving distance of each object; The relative moving speed of each of the objects is acquired according to the image acquisition time of the current surround view image, the image acquisition time of each of the historical surround view images, and the moving distances of all the objects.

4. The method according to claim 3, characterized in that The acquiring of a plurality of continuous surround view images corresponding to the current surround view image comprises: The image acquisition time of the current surround view image is read, and a plurality of continuous historical surround view images corresponding to the image acquisition time are acquired from a database according to a preset time window threshold.

5. The method according to claim 1, characterized in that The method of obtaining the current surround view image of the vehicle through a preset surround view system, identifying all objects in the current surround view image, and obtaining the object category of each object includes: Based on multiple image acquisition devices deployed at the front, left, right and rear of the vehicle, multiple image data of the vehicle's surrounding environment are collected and acquired according to a preset sampling frequency; According to the multiple image data collected at the same time, an image stitching algorithm is used to obtain a stitching image of the surrounding environment of the vehicle; Performing image correction on the stitched image to obtain the current surround image; wherein the image stitching algorithm includes one of feature matching stitching, transformation stitching or a deep learning-based stitching algorithm; According to the current surround image, a region growing algorithm is used to segment the current surround image to obtain a plurality of segmented images; According to the plurality of segmented images, feature extraction is performed on the segmented images to obtain feature information of each segmented image; According to all the feature information, a support vector machine classification algorithm is used to identify the object corresponding to each feature information, and obtain the object category corresponding to the object.

6. The method according to claim 1 or 2, characterized in that: The step of obtaining the weighted feature value of the object according to the relative moving speed and the object category of the object and the number of objects contained in the current surround image comprises: According to the object category of the object, query and obtain the category parameter and the category weight value corresponding to the object category, and calculate and obtain the category weighted feature value through weighted summation based on the category parameters and the category weight values ​​of all the objects; According to the relative moving speed of the object, query and obtain a speed weighted value corresponding to the relative moving speed, and calculate and obtain a speed weighted characteristic value by weighted summation based on the relative moving speed and the speed weighted value; According to the category weighted feature value, the speed weighted feature value and the number of objects, the object weighted feature value corresponding to the current environment is obtained by preset weight threshold.

7. The method according to claim 1 or 2, characterized in that: The current driving environment information includes at least one of ground friction coefficient, light intensity and rainfall information; Then, according to the current driving environment information of the vehicle, a driving environment weighted characteristic value is obtained, including: Obtain at least one of the ground friction coefficient, light intensity and rainfall information of the current environment; According to at least one of the ground friction coefficient, light intensity and rainfall information, query and obtain a weighted value corresponding to at least one of the ground friction coefficient, light intensity and rainfall information; According to at least one of the ground friction coefficient, light intensity and rainfall information, and a weighted value corresponding to at least one of the ground friction coefficient, light intensity and rainfall information, a weighted characteristic value of the driving environment is calculated by weighted summation.

8. The method according to claim 1 or 2, characterized in that: The current running state information includes at least one of vehicle speed, steering state and yaw angle; Then, according to the current running state information of the vehicle, a weighted characteristic value of the running state is obtained, including: According to the current running state of the vehicle, obtaining at least one of the vehicle speed, the steering state and the yaw angle; According to at least one of the vehicle speed, the steering state and the yaw angle, query and obtain a weighted value corresponding to at least one of the vehicle speed, the steering state and the yaw angle; The weighted characteristic value of the running state is calculated and obtained by weighted summation based on at least one of the vehicle speed, the steering state and the yaw angle, and a weighted value corresponding to at least one of the vehicle speed, the steering state and the yaw angle.

9. The method according to any one of claims 1 to 5, characterized in that: After identifying all objects in the current surround image and obtaining the object category of each of the objects, the method further includes: According to the object category, a color frame matching the object category is used to select the object in the current surround view image to obtain a marked surround view image; The marked surround view image is sent to the central control screen of the vehicle so that the central control screen can visualize the marked surround view image.

10. An early warning device, characterized in that: include: An object recognition module, used to obtain a current surround view image of the vehicle through a preset surround view system, identify all objects in the current surround view image, and obtain an object category of each of the objects; a speed calculation module, configured to obtain a plurality of continuous historical surround view images corresponding to the current surround view image, and based on all objects in the current surround view image, obtain a relative moving speed between each object and the vehicle according to the current surround view image and the plurality of continuous historical surround view images; A weighted feature calculation module, used to obtain a weighted feature value of the object according to the relative moving speed and the object category of the object and the number of objects contained in the current surround image, and to obtain a weighted feature value of the running state according to the current running state information of the vehicle, and to obtain a weighted feature value of the driving environment according to the current driving environment information of the vehicle; The warning execution module is used to obtain the warning strategy according to the weighted feature value of the object, the weighted feature value of the operating state and the weighted feature value of the driving environment, using the trained warning strategy classification model, and complete the blind spot warning according to the warning strategy.

11. A vehicle, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 9 when executed by a processor.

Images