Extreme scene recognition method, device and equipment and automobile

Abstract

The invention relates to an extreme scene recognition method, device and equipment and an automobile, and the method comprises the steps: obtaining a to-be-recognized video of the surrounding environment of the automobile in the automatic driving process of the automobile; inputting the to-be-identified video into a pre-constructed video understanding large model for processing to obtain video understanding information of the to-be-identified video; video understanding information indicating at least one extreme scene object; the extreme scene object is a video object used for identifying whether the video comprises an extreme scene; according to the video understanding information, extreme scene recognition processing is carried out on the to-be-recognized video, and an extreme scene recognition result is obtained and displayed; wherein the extreme scene recognition result is used for indicating whether the to-be-recognized video comprises an extreme scene or not. In the process of identifying the extreme scene, whether the to-be-identified video comprises the extreme scene is identified based on the video understanding information of the to-be-identified video, so that the accuracy and the efficiency of identifying the extreme scene can be improved.

Description

Technical Field

[0001] This invention relates to the field of automotive technology, and more specifically to a method, apparatus, device, and vehicle for identifying extreme scenarios. Background Technology

[0002] With the development of vehicle technology, autonomous driving functions are being used more and more frequently during vehicle operation. Among these challenges, effectively handling extreme scenarios has become one of the key technical difficulties in autonomous driving. However, identifying extreme scenarios is crucial before attempting to address them.

[0003] In existing technologies, extreme scenarios are mainly identified through manual methods or preset recognition rules.

[0004] However, existing technologies can lead to inaccurate and inefficient identification. Summary of the Invention

[0005] One objective of this invention is to provide a method for identifying extreme scenarios, thereby solving the problems of low efficiency and low accuracy in the identification of extreme scenarios in the prior art; another objective is to provide a device for identifying extreme scenarios; a third objective is to provide an electronic device; a fourth objective is to provide a computer-readable storage medium; a fifth objective is to provide a computer program product; and a sixth objective is to provide an automobile.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A method for identifying extreme scenarios, comprising:

[0008] During the autonomous driving process of the vehicle, a video of the surrounding environment of the vehicle to be identified is acquired;

[0009] The video to be identified is input into a pre-constructed large-scale video understanding model for processing to obtain video understanding information of the video to be identified; wherein, the large-scale video understanding model is a model for understanding and describing the video to be identified; the video understanding information indicates at least one extreme scene object; the extreme scene object refers to a video object used to identify whether the video includes an extreme scene;

[0010] Based on the video understanding information, extreme scene recognition processing is performed on the video to be identified, and the extreme scene recognition result is obtained and displayed; wherein, the extreme scene recognition result is used to indicate whether the video to be identified includes extreme scenes.

[0011] Based on the aforementioned technical means, in the process of identifying extreme scenes, the identification of whether the video to be identified contains extreme scenes based on the video understanding information of the video to be identified can improve the comprehensiveness and accuracy of extreme scene identification. In the process of determining the video understanding information of the video to be identified, the pre-built large model can improve the generation efficiency and accuracy of video understanding information, further improving the accuracy and efficiency of extreme scene identification. In summary, the accuracy and efficiency of extreme scene identification are improved.

[0012] Furthermore, the construction process of the large-scale video understanding model includes: obtaining an initial large-scale video understanding model and labeled historical videos; wherein, the initial large-scale video understanding model is an initial model constructed based on a convolutional neural network and a large language model; wherein, the labeled historical videos include historical videos and annotation information; the historical videos are videos of the surrounding environment acquired during the vehicle's historical autonomous driving process; the annotation information is historical video understanding information corresponding to the historical information; and training the initial large-scale video understanding model based on the labeled historical videos to obtain the large-scale video understanding model.

[0013] Based on the aforementioned technical means, the initial video understanding big model is an initial model constructed based on convolutional neural networks and a large language model. Therefore, the video understanding big model constructed based on the initial video understanding big model can accurately extract video feature information of the video to be identified, and can also accurately understand the video feature information, thereby improving the accuracy of the obtained video understanding information and thus providing accuracy for extreme scene recognition. Among them, the historical videos in the labeled historical videos are videos of the surrounding environment acquired during the vehicle's historical autonomous driving process, which are easy to obtain. Therefore, the training set of the initial video understanding big model is richer, thereby improving the accuracy of the trained video understanding big model and further improving the accuracy of extreme scene recognition.

[0014] Furthermore, the initial video understanding model includes an initial video feature extraction layer, a prompt word generation layer, and a video feature understanding layer; wherein, the initial video feature extraction layer is a network layer constructed based on a convolutional neural network to extract video features; the prompt word generation layer is a pre-constructed network layer that converts video feature information into video feature prompt words; and the video feature understanding layer is a network layer constructed based on a base large language model for generating video understanding information; training the initial video understanding model based on the labeled historical videos to obtain the video understanding model includes: inputting historical videos from the labeled historical videos into the initial video feature extraction layer included in the initial video understanding model to obtain corresponding predicted video feature information; and training the initial video understanding model based on the prompt words included in the initial video understanding model. The generation layer processes the predicted video feature information to generate corresponding historical video feature understanding prompts. These historical video feature understanding prompts are then input into the video feature understanding layer within the initial video understanding model to obtain corresponding predicted video understanding information. Based on the predicted video understanding information and the annotation information included in the labeled historical videos, the parameters and weights of the initial video feature extraction layer are adjusted to obtain the trained initial video understanding model. If the trained initial video understanding model meets a preset convergence condition, it is determined as the video understanding model. Otherwise, the process of training the initial video understanding model based on the labeled historical videos is repeated until the trained initial video understanding model meets the preset convergence condition.

[0015] Based on the aforementioned technical methods, in the process of training an initial large-scale video understanding model using labeled historical videos, since the video feature understanding layer is a base large-scale language model, there is no need to adjust its corresponding parameters and weights during training. Only the parameters and weights of the video feature extraction layer need to be adjusted, which improves the efficiency of model training. Furthermore, because the base large-scale speech model has strong understanding capabilities, the initial large-scale video understanding model constructed has strong video understanding capabilities, resulting in a strong video understanding capability for the trained large-scale video understanding model. Therefore, the accuracy of determining video understanding information based on the model is improved, further enhancing the accuracy of extreme scene recognition. In the process of developing the large-scale video understanding model, the labeled historical videos used are not limited to those involving extreme scenes. This increases the number of training samples for model training, improving the accuracy of model training and the accuracy of the trained video understanding model, further enhancing the accuracy of determining video understanding information based on the model.

[0016] Furthermore, based on the video understanding information, extreme scene recognition processing is performed on the video to be recognized to obtain and display the extreme scene recognition result, including: determining at least one extreme scene object and the relative relationship between each extreme scene object based on the video understanding information; determining whether the video to be recognized includes an extreme scene based on the at least one extreme scene object and the relative relationship between each extreme scene object; if it is determined that the video to be recognized includes an extreme scene, then determining the scene type, scene location, scene timestamp, and scene severity level of the extreme scene, and generating and displaying the corresponding extreme scene recognition result based on the scene type, scene location, scene timestamp, and scene severity level of the extreme scene; otherwise, directly generating and displaying the corresponding extreme scene recognition result.

[0017] Based on the aforementioned technical means, in the process of obtaining and displaying the extreme scene recognition results, the determination of whether the video to be recognized includes extreme scenes can be accurately determined by identifying the extreme scene objects and the relative relationships between them. In particular, when it is determined that the video to be recognized includes extreme scenes, the extreme scene recognition results can be determined and displayed by determining the scene type, scene location, scene timestamp, and scene severity level of the extreme scene, which can improve the intuitiveness of the results presentation.

[0018] Furthermore, after acquiring the video to be identified, the method further includes: determining the environmental information and scene information of the video to be identified; determining a corresponding video preprocessing strategy based on the environmental information and the scene information; and preprocessing the video to be identified according to the corresponding video preprocessing strategy.

[0019] Based on the above technical means, after acquiring the video to be identified, preprocessing the video can improve the accuracy of the video to be identified, and further improve the accuracy of extreme scene identification based on the video to be identified. In the process of preprocessing the video to be identified, determining the corresponding video preprocessing strategy according to the environmental information and scene information corresponding to the video to be identified can further improve the accuracy of video preprocessing, and also further improve the accuracy of extreme scene identification.

[0020] Furthermore, the method also includes: if the extreme scene recognition result indicates that the video to be recognized includes an extreme scene, then perform autonomous driving optimization processing on the video to be recognized.

[0021] Based on the aforementioned technical means, after determining that the video to be identified includes extreme scenes, the autonomous driving optimization process of the vehicle can be performed based on the video to be identified, which can improve the accuracy and safety of autonomous driving.

[0022] An extreme scene recognition device, comprising:

[0023] The acquisition module is used to acquire a video of the surrounding environment of the vehicle to be identified during the autonomous driving process of the vehicle.

[0024] The processing module is used to input the video to be identified into a pre-constructed video understanding model for processing to obtain video understanding information of the video to be identified; wherein, the video understanding model is a model for understanding and describing the video to be identified; the video understanding information indicates at least one extreme scene object; the extreme scene object refers to a video object used to identify whether the video includes an extreme scene;

[0025] The recognition module is used to perform extreme scene recognition processing on the video to be recognized based on the video understanding information, and to obtain and display the extreme scene recognition result; wherein the extreme scene recognition result is used to indicate whether the video to be recognized includes an extreme scene.

[0026] Furthermore, the device also includes a construction module for acquiring an initial large-scale video understanding model and labeled historical videos; wherein the initial large-scale video understanding model is an initial model constructed based on a convolutional neural network and a large language model; wherein the labeled historical videos include historical videos and annotation information; the historical videos are videos of the surrounding environment acquired during the vehicle's historical autonomous driving process; the annotation information is historical video understanding information corresponding to the historical information; and the initial large-scale video understanding model is trained based on the labeled historical videos to obtain the large-scale video understanding model.

[0027] Furthermore, the initial video understanding model includes an initial video feature extraction layer, a prompt word generation layer, and a video feature understanding layer. The initial video feature extraction layer is a network layer constructed based on a convolutional neural network to extract video features. The prompt word generation layer is a pre-constructed network layer that converts video feature information into video feature prompt words. The video feature understanding layer is a network layer constructed based on a base large language model for generating video understanding information. The construction module is specifically used to input historical videos from the labeled historical videos into the initial video feature extraction layer included in the initial video understanding model to obtain corresponding predicted video feature information; and to process the predicted video feature information according to the prompt word generation layer included in the initial video understanding model. The system generates prompt words to obtain corresponding historical video feature understanding prompt words. These prompt words are then input into the video feature understanding layer within the initial video understanding model to obtain corresponding predicted video understanding information. Based on the predicted video understanding information and the annotation information included in the labeled historical videos, the parameters and weights of the initial video feature extraction layer are adjusted to obtain the trained initial video understanding model. If the trained initial video understanding model meets a preset convergence condition, it is determined as the video understanding model. Otherwise, the process of training the initial video understanding model based on the labeled historical videos is repeated until the trained initial video understanding model meets the preset convergence condition.

[0028] Furthermore, the recognition module is specifically used to determine at least one extreme scene object and the relative relationship between each extreme scene object based on the video understanding information; determine whether the video to be recognized includes an extreme scene based on the at least one extreme scene object and the relative relationship between each extreme scene object; if it is determined that the video to be recognized includes an extreme scene, then determine the scene type, scene location, scene timestamp, and scene severity level of the extreme scene, and generate and display the corresponding extreme scene recognition result based on the scene type, scene location, scene timestamp, and scene severity level of the extreme scene; otherwise, directly generate and display the corresponding extreme scene recognition result.

[0029] Furthermore, the processing module is also used to determine the environmental information and scene information of the video to be identified; determine the corresponding video preprocessing strategy based on the environmental information and the scene information; and preprocess the video to be identified according to the corresponding video preprocessing strategy.

[0030] Furthermore, the extreme scene recognition device also includes an application module, which is used to perform autonomous driving optimization processing on the video to be recognized if the extreme scene recognition result indicates that the video to be recognized includes an extreme scene.

[0031] An electronic device, including a memory and a processor;

[0032] The memory stores computer-executed instructions;

[0033] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0034] A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible embodiments of the first aspect as described above.

[0035] A computer program product includes a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect as described above.

[0036] An automobile includes electronic devices that implement the first aspect and / or various possible embodiments of the first aspect as described above.

[0037] The beneficial effects of this invention are:

[0038] The extreme scene recognition method provided in this invention involves acquiring a video of the vehicle's surrounding environment during autonomous driving, inputting the video into a pre-built video understanding model for processing to obtain video understanding information, and then performing extreme scene recognition processing on the video based on the video understanding information to obtain and display the extreme scene recognition result. The video understanding model is a model that understands and describes the video; the video understanding information indicates at least one extreme scene object; and the extreme scene object refers to a video object used to identify whether the video includes an extreme scene. In the process of extreme scene recognition, identifying whether the video includes an extreme scene based on the video understanding information can improve the comprehensiveness and accuracy of extreme scene recognition. Furthermore, in determining the video understanding information of the video, the pre-built model can improve the generation efficiency and accuracy of the video understanding information, further improving the accuracy and efficiency of extreme scene recognition. In summary, this method improves the accuracy and efficiency of extreme scene recognition. Attached Figure Description

[0039] Figure 1The flowchart of an extreme scene recognition method provided in an embodiment of the present invention Figure 1 ;

[0040] Figure 2 The flowchart of an extreme scene recognition method provided in an embodiment of the present invention Figure 2 ;

[0041] Figure 3 The flowchart of an extreme scene recognition method provided in an embodiment of the present invention Figure 3 ;

[0042] Figure 4 A schematic diagram of the structure of an extreme scene recognition device provided in an embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0044] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0045] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0046] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0047] In existing technologies, extreme scenarios are primarily identified manually or through pre-defined recognition rules. However, these methods often result in inaccurate and inefficient identification.

[0048] This application acquires a video of the vehicle's surrounding environment during autonomous driving, inputs the video into a pre-built video understanding model for processing, and obtains video understanding information for the video. Based on the video understanding information, extreme scene recognition processing is performed on the video to be recognized, and the extreme scene recognition result is obtained and displayed. Here, the video understanding model is a model that understands and describes the video to be recognized; the video understanding information indicates at least one extreme scene object; the extreme scene object refers to a video object used to identify whether the video includes an extreme scene. In the process of extreme scene recognition, identifying whether the video to be recognized includes an extreme scene based on the video understanding information can improve the comprehensiveness and accuracy of extreme scene recognition. Furthermore, in determining the video understanding information of the video to be recognized, the pre-built model can improve the efficiency and accuracy of video understanding information generation, further improving the accuracy and efficiency of extreme scene recognition. In summary, the accuracy and efficiency of extreme scene recognition are improved.

[0049] The technical solution of the present invention will now be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0050] Figure 1 The flowchart of an extreme scene recognition method provided in an embodiment of the present invention Figure 1 .like Figure 1 As shown, the method of this embodiment of the invention includes:

[0051] Step S101: During the autonomous driving process of the vehicle, acquire the video of the surrounding environment of the vehicle to be identified.

[0052] Specifically, during the autonomous driving process of the vehicle, a video of the vehicle's surrounding environment to be identified can be acquired. This application does not limit the process of acquiring this video; optionally, the video can be acquired from at least one video acquisition device. Optionally, the video acquisition device can be a camera positioned at a preset location on the vehicle. This application does not limit the preset location of the video acquisition device; any preset location capable of acquiring the video of the vehicle's surrounding environment can be used as the preset location provided in this application. Optionally, the preset location can be the front, rear, or side of the vehicle.

[0053] Optionally, after acquiring videos of the vehicle's surrounding environment from each video acquisition device, time synchronization processing can be performed on each acquired video based on time information, and image stitching processing can be performed on the time-synchronized videos to obtain the video of the vehicle's surrounding environment to be identified, thereby improving the accuracy of the acquired video of the vehicle's surrounding environment to be identified.

[0054] Step S102: Input the video to be recognized into the pre-built video understanding big model for processing to obtain the video understanding information of the video to be recognized.

[0055] Specifically, the video to be identified obtained in step S101 can be input into a pre-built large-scale video understanding model for processing to obtain video understanding information of the video to be identified.

[0056] The video understanding big model is a model that understands and describes the video to be recognized. This application does not limit the video understanding big model; any model capable of understanding and describing the video to be recognized can be used as the video understanding big model provided in this application. Optionally, the video understanding big model can be a model trained based on labeled historical videos and an initial video understanding big model constructed using convolutional neural networks and a large language model.

[0057] Among them, video understanding information indicates at least one extreme scene object; an extreme scene object refers to a video object used to identify whether the video includes an extreme scene;

[0058] The term "corner case" in this context refers to uncommon or extreme situations in autonomous driving scenarios that are not typical under normal driving conditions. These situations may lead to unexpected system behavior or malfunctions. This application does not limit the definition of "corner case"; any scenario that could cause unexpected system behavior or malfunctions can be considered an extreme case provided in this application. Optionally, extreme cases include, but are not limited to: complex weather conditions, rare traffic situations, irregular pedestrian behavior, and abnormal road conditions. Complex weather conditions include driving in extreme weather conditions such as blizzards or dense fog; rare traffic situations include, for example, encountering a sudden traffic accident or obstacle at a sharp turn; irregular pedestrian behavior includes, for example, a pedestrian suddenly crossing the road on a non-pedestrian crossing; and abnormal road conditions include, for example, road construction, road surface damage, or sudden roadblocks.

[0059] This application does not limit the extreme scene object. Any video object used to identify whether a video includes an extreme scene can be used as the extreme scene object provided in this application. Optionally, the extreme scene object includes, but is not limited to, video objects such as vehicle objects, pedestrian objects, and traffic sign objects.

[0060] Step S103: Based on the video understanding information, perform extreme scene recognition processing on the video to be recognized, and obtain and display the extreme scene recognition results.

[0061] Specifically, based on the video understanding information obtained in step S102, extreme scene recognition processing can be performed on the video to be recognized, and the extreme scene recognition result can be obtained and displayed. The extreme scene recognition result refers to the result of identifying whether the scene described in the video understanding information includes the extreme scene described above. The extreme scene recognition result is used to indicate whether the video to be recognized includes an extreme scene. This application does not limit the composition of the extreme scene recognition result. Optionally, if the extreme scene recognition result indicates that the video to be recognized includes an extreme scene, the extreme scene recognition result can be composed of information such as the extreme scene type, extreme scene location, extreme scene timestamp, and severity level of the extreme scene.

[0062] This application does not limit the process of performing extreme scene recognition processing on the video to be recognized based on video understanding information, and obtaining and displaying the extreme scene recognition result. Optionally, it can first determine at least one extreme scene object and the relative relationship between each extreme scene object based on the video understanding information, and then determine whether the video to be recognized includes an extreme scene based on the at least one extreme scene object and the relative relationship between each extreme scene object. If it is determined that the video to be recognized includes an extreme scene, then the scene type, scene location, scene timestamp, and scene severity level of the extreme scene are determined, and the corresponding extreme scene recognition result is generated and displayed based on the scene type, scene location, scene timestamp, and scene severity level of the extreme scene; otherwise, the corresponding extreme scene recognition result is directly generated and displayed.

[0063] This application does not limit the device for displaying extreme scene recognition results; any device capable of displaying extreme scene recognition results can be used as the device provided in this application for displaying extreme scene recognition results. Optionally, it can be displayed in a vehicle's in-vehicle display device, in a user's mobile terminal, or in a virtual reality headset.

[0064] The extreme scene recognition method provided in this invention involves acquiring a video of the vehicle's surrounding environment during autonomous driving, inputting the video into a pre-built video understanding model for processing to obtain video understanding information, and then performing extreme scene recognition processing on the video based on the video understanding information to obtain and display the extreme scene recognition result. The video understanding model is a model that understands and describes the video; the video understanding information indicates at least one extreme scene object; and the extreme scene object refers to a video object used to identify whether the video includes an extreme scene. In the process of extreme scene recognition, identifying whether the video includes an extreme scene based on the video understanding information can improve the comprehensiveness and accuracy of extreme scene recognition. Furthermore, in determining the video understanding information of the video, the pre-built model can improve the generation efficiency and accuracy of the video understanding information, further improving the accuracy and efficiency of extreme scene recognition. In summary, this method improves the accuracy and efficiency of extreme scene recognition.

[0065] In one possible embodiment, after acquiring the video to be identified, the extreme scene identification method may further include: first determining the environmental and scene information of the video to be identified; then determining a corresponding video preprocessing strategy based on the environmental and scene information; and finally, preprocessing the video to be identified according to the corresponding video preprocessing strategy.

[0066] The environmental information refers to information about the environment in which the vehicle is located during autonomous driving. This application does not limit the scope of environmental information; any information that can describe the environment in which the vehicle is located during autonomous driving can be used as the environmental information provided in this application. Optionally, the environmental information includes, but is not limited to: road environmental information, obstacle environmental information, natural environmental information, and sensor environmental information. Road environmental information includes, but is not limited to: road type, road signs, lane information, etc.; obstacle environmental information includes, but is not limited to: fixed obstacle information, static obstacle information, dynamic obstacle information, etc.; natural environmental information includes, but is not limited to: weather conditions information, lighting conditions information, etc.; and sensor environmental information includes, but is not limited to: LiDAR information, camera information, etc.

[0067] The "scene information" refers to information about the scenario in which the vehicle is located during autonomous driving. This application does not limit the scope of the scene information; any information that can describe the scenario in which the vehicle is located during autonomous driving can be used as the scene information provided in this application. Optionally, the scene information includes, but is not limited to, driving scene information and traffic rule scene information. Driving scene information includes, but is not limited to, intersection information, roundabout information, and parking information; traffic rule scene information includes, but is not limited to, traffic light rule information, zebra crossing rule information, yielding rule information, lane changing rule information, parking rule information, and overtaking rule information.

[0068] This application does not limit the process of determining the environmental and scene information of the video to be identified. Optionally, feature information corresponding to the environmental information and feature information corresponding to the scene information can be extracted from the video to be identified, and then the environmental and scene information of the video to be identified can be determined based on the corresponding feature information. This application does not limit the process of determining the corresponding video preprocessing strategy based on the environmental and scene information. Optionally, the corresponding video preprocessing strategy can be obtained from a preset correspondence between video preprocessing strategies based on the environmental and scene information, wherein the environmental information, scene information, and video preprocessing strategy correspond one-to-one in the preset correspondence between video preprocessing strategies.

[0069] The video preprocessing strategy is a strategy for preprocessing the video to be processed. This application does not limit the video preprocessing strategy. Optionally, the video preprocessing strategy may include at least one video preprocessing process. This application does not limit the video preprocessing process. Optionally, the video preprocessing process may include any one of the following processing processes: image scaling, image normalization, image denoising, image enhancement, etc.

[0070] Among them, after acquiring the video to be identified, preprocessing the video can improve the accuracy of the video to be identified, and further improve the accuracy of extreme scene identification based on the video to be identified. In the process of preprocessing the video to be identified, determining the corresponding video preprocessing strategy according to the environmental and scene information corresponding to the video to be identified can further improve the accuracy of video preprocessing, and also further improve the accuracy of extreme scene identification.

[0071] In one possible embodiment, the extreme scene identification method further includes: if the extreme scene identification result indicates that the video to be identified includes an extreme scene, then performing autonomous driving optimization processing on the video to be identified.

[0072] This application does not limit the process of optimizing autonomous driving based on the video to be identified. Any process that can optimize autonomous driving based on the video to be identified can be used as the process provided in this application. Optionally, the video to be identified can be labeled based on the extreme scene recognition results. Then, the training set of the autonomous driving perception model can be updated based on the labeled video. Finally, the autonomous driving perception model can be trained based on the updated training set to optimize the autonomous driving of the vehicle.

[0073] In particular, after determining that the video to be identified contains extreme scenes, the autonomous driving optimization process of the vehicle is performed based on the video to be identified, which can improve the accuracy and safety of autonomous driving.

[0074] In one possible embodiment, if the extreme scene identification method further includes: if the extreme scene identification result indicates that the video to be identified includes an extreme scene, then performing autonomous driving optimization processing on the video to be identified. This means that after acquiring videos of the vehicle's surrounding environment, videos corresponding to abnormal driving situations can be selected and used as the videos to be identified. Wherein, if the vehicle's autonomous driving is an abnormal driving situation, the corresponding video is more likely to include extreme scenes. Therefore, the efficiency of extreme scene identification can be improved, further enhancing the efficiency of autonomous driving optimization processing.

[0075] Figure 2 The flowchart of an extreme scene recognition method provided in an embodiment of the present invention Figure 2 .exist Figure 1 Based on the illustrated embodiment, this embodiment describes in detail the process of performing extreme scene recognition processing on the video to be recognized based on video understanding information, and obtaining and displaying the extreme scene recognition results, as follows: Figure 2 As shown, the method of this embodiment of the invention includes:

[0076] Step S201: Based on the video understanding information, determine at least one extreme scene object and the relative relationships between each extreme scene object.

[0077] Specifically, at least one extreme scene object and the relative relationships between these extreme scene objects can be determined based on video understanding information. The description of the video understanding information and the extreme scene objects can be found in step S102 and will not be repeated here. The relative relationships between the extreme scene objects are the relative relationships between any two of the at least one determined extreme scene objects. This application does not limit the relative relationships; optionally, the relative relationships include, but are not limited to, at least one of the following: relative distance, relative position, relative motion, and relative time.

[0078] Optionally, the video understanding information may include descriptive information of at least one extreme scene object. This application does not limit the descriptive information of the extreme scene object; any information capable of describing the extreme scene object can be used as the descriptive information of the extreme scene object provided in this application. Optionally, the descriptive information of the extreme scene object includes, but is not limited to: type description information, color description information, shape description information, time description information, motion trajectory description information, position description information, etc.

[0079] In this application, the process of determining at least one extreme scene object and the relative relationship between each extreme scene object based on video understanding information is not limited. Optionally, if, as described above, the video understanding information includes descriptive information of at least one extreme scene object, then the process of determining at least one extreme scene object and the relative relationship between each extreme scene object based on the video understanding information may include: determining at least one extreme scene object and the relative relationship between each extreme scene object based on the descriptive information of at least one extreme scene object included in the video understanding information.

[0080] Optionally, if the description information of the extreme scene objects includes: type description information, color description information, shape description information, time description information, motion trajectory description information, position description information, etc., the process of determining at least one extreme scene object and the relative relationship between each extreme scene object based on the description information of at least one extreme scene object included in the video understanding information may include: determining at least one extreme scene object based on the type description information, color description information, shape description information, etc. included in the description information of the extreme scene objects; and determining the relative relationship between each extreme scene object based on the time description information, motion trajectory description information, position description information, etc. included in the description information of the extreme scene objects.

[0081] Step S202: Determine whether the video to be identified includes an extreme scene based on at least one extreme scene object and the relative relationships between the extreme scene objects.

[0082] Specifically, based on at least one extreme scene object determined in step S201 and the relative relationships between each extreme scene object, it can be determined whether the video to be identified includes an extreme scene.

[0083] This application does not limit the process of determining whether a video to be identified includes an extreme scene based on at least one extreme scene object and the relative relationships between these extreme scene objects. Optionally, it can determine whether a video to be identified includes an extreme scene based on at least one extreme scene object, the relative relationships between these extreme scene objects, and a preset mapping relationship. The preset mapping relationship is a mapping relationship between extreme scenes, extreme scene objects, and the relative relationships between extreme scene objects; that is, a preset extreme scene object is determined to be an extreme scene if it satisfies the preset relative relationships between extreme scene objects.

[0084] Step S203: If it is determined that the video to be identified includes an extreme scene, then determine the scene type, scene location, scene timestamp, and severity level of the extreme scene, and generate and display the corresponding extreme scene recognition result based on the scene type, scene location, scene timestamp, and severity level of the extreme scene; otherwise, directly generate and display the corresponding extreme scene recognition result.

[0085] Specifically, according to the process in step S202, if it is determined that the video to be identified includes an extreme scene, the scene type, scene location, scene timestamp, and severity level of the extreme scene can be determined. Based on these parameters, a corresponding extreme scene identification result is generated and displayed. This result indicates that the video to be identified includes an extreme scene, and also indicates the scene information such as the scene type, scene location, scene timestamp, and severity level of the included extreme scene. If it is determined that the video to be identified includes an extreme scene, and the number of included extreme scenes is greater than or equal to one, then the corresponding extreme scene identification result indicates that the video to be identified includes at least one extreme scene, and also indicates the scene information such as the scene type, scene location, scene timestamp, and severity level of each included extreme scene.

[0086] Specifically, according to the process in step S202, if it is determined that the video to be identified does not contain extreme scenes, then the corresponding extreme scene identification result is directly generated and displayed. The corresponding extreme scene identification result indicates that the video to be identified does not contain extreme scenes.

[0087] The present invention provides a process for identifying extreme scenes in a video based on video understanding information, and obtaining and displaying the extreme scene identification results. This process involves determining at least one extreme scene object and the relative relationships between these objects based on the video understanding information. Based on these objects, it determines whether the video contains an extreme scene. If an extreme scene is found, the scene type, location, timestamp, and severity level are determined. The corresponding extreme scene identification result is then generated and displayed based on these parameters. Otherwise, the result is directly generated and displayed. In obtaining and displaying the extreme scene identification results, the determination of the extreme scene object and its relative relationships allows for accurate identification of whether the video contains an extreme scene. Furthermore, when an extreme scene is determined, the determination of the scene type, location, timestamp, and severity level improves the intuitiveness of the result presentation.

[0088] Figure 3 The flowchart of an extreme scene recognition method provided in an embodiment of the present invention Figure 3 .exist Figure 1 or Figure 2 Based on the illustrated embodiment, this embodiment provides a detailed description of the construction process of the large-scale video understanding model. For example... Figure 3 As shown, the method of this embodiment of the invention includes:

[0089] Step S301: Obtain the initial video understanding model and the labeled historical videos.

[0090] Specifically, it can obtain the initial video understanding model and the labeled historical videos.

[0091] The initial video understanding model is the initial model used to train the large-scale video understanding model. This application does not limit the initial video understanding model; any initial model used for training the large-scale video understanding model can be used as the initial video understanding model provided in this application. Optionally, the initial video understanding model is an initial model constructed based on a convolutional neural network and a large language model.

[0092] The labeled historical videos include historical video information and labeling information. The historical videos are videos of the surrounding environment acquired during the vehicle's historical autonomous driving process. This application does not limit whether the historical videos include extreme scenes; optionally, the historical videos may or may not include extreme scenes. The labeling information is the historical video understanding information corresponding to the historical information. The description of the historical video understanding information itself can refer to the description of video understanding information in step S102, that is, the historical video understanding information indicates at least one extreme scene object; here, the extreme scene object refers to the video object used to determine whether the historical video includes extreme scenes.

[0093] Step S302: Train the initial video understanding model based on the labeled historical videos to obtain the video understanding model.

[0094] Specifically, based on the labeled historical videos obtained in step S301, the initial video understanding big model obtained in step S301 is trained to obtain the video understanding big model.

[0095] This application does not limit the process of training an initial large-scale video understanding model based on labeled historical videos to obtain the large-scale video understanding model. Optionally, the initial large-scale video understanding model includes an initial video feature extraction layer, a cue word generation layer, and a video feature understanding layer. The initial video feature extraction layer is a network layer constructed based on a convolutional neural network to extract video features; the cue word generation layer is a pre-constructed network layer that converts video feature information into video feature cue words; and the video feature understanding layer is a network layer constructed based on a base large-scale language model to generate video understanding information.

[0096] Optionally, if the initial video understanding model includes an initial video feature extraction layer, a cue word generation layer, and a video feature understanding layer, then the process of training the initial video understanding model based on labeled historical videos to obtain the final video understanding model may include:

[0097] First, the historical videos from the labeled historical videos are input into the initial video feature extraction layer included in the initial video understanding model to obtain the corresponding predicted video feature information.

[0098] Among them, the predicted video feature information is the information obtained by the initial video feature extraction layer in the initial video understanding model after extracting features from historical videos.

[0099] Secondly, based on the prompt word generation layer included in the initial video understanding model, prompt words are generated from the predicted video feature information to obtain the corresponding historical video feature understanding prompt words. These historical video feature understanding prompt words are then input into the video feature understanding layer included in the initial video understanding model to obtain the corresponding predicted video understanding information.

[0100] The process of generating prompts for predicted video features based on the prompt generation layer included in the initial video understanding model can be described as follows: the predicted video features are processed into prompts, which are then input into the prompt generation layer included in the initial video understanding model, and the corresponding historical video feature understanding prompts are output.

[0101] The historical video feature understanding prompts are prompts used to understand and describe the video feature information of historical videos, such as the predicted video feature information described above. These historical video feature understanding prompts can be a combination of a preset prompt generation template and the predicted video feature information. Optionally, the preset prompt generation template indicates the constraints for generating the predicted video understanding information. The resulting historical video feature understanding information is the information obtained after understanding and describing the predicted video feature information according to the constraints of the predicted video understanding information. Optionally, the preset prompt generation template can also indicate the composition of the predicted video understanding information, the format of the video understanding information, and other information.

[0102] The video feature understanding layer included in the initial large-scale video understanding model can serve as a foundational large-scale language model. A foundational large-scale language model typically refers to a large-scale pre-trained language model trained on massive amounts of text data, capable of understanding and generating natural language. The "foundation" part of these models means they can serve as a base model for further fine-tuning or customization for specific tasks. These models are typically designed to capture complex patterns and structures in language, enabling them to excel in various natural language processing tasks, such as text generation, translation, question answering, and sentiment analysis. Typical examples of large-scale language models are OpenAI's GPT (Generative Pre-trained Transformer) series and Google's BERT (Bidirectional Encoder Representations from Transformers). These models typically have billions or even hundreds of billions of parameters, and through training on large-scale text datasets, they learn the syntax, semantics, and contextual relationships of language. The advantage of foundational large-scale language models lies in their versatility and powerful transfer learning capabilities. With minimal fine-tuning on specific tasks, they can achieve high performance in various application scenarios. This makes them highly adaptable to various tasks.

[0103] Then, based on the predicted video understanding information and the annotation information included in the labeled historical videos, the parameters and weights of the initial video feature extraction layer are adjusted to obtain the trained initial video understanding model.

[0104] Specifically, the purpose of adjusting the parameters and weights of the initial video feature extraction layer is to make the predicted video understanding information obtained based on the trained initial video understanding model more closely match the annotation information included in the labeled historical videos.

[0105] Finally, if the trained initial video understanding model meets the preset convergence criteria, then the trained initial video understanding model is determined as the video understanding model. Otherwise, the process of training the initial video understanding model based on the labeled historical videos is repeated until the trained initial video understanding model meets the preset convergence criteria.

[0106] The preset convergence condition is a preset condition that the large-scale video understanding model has completed training. The application does not limit the preset convergence condition; any preset condition that the large-scale video understanding model has completed training can be used as the preset convergence condition provided in this application. Optionally, it can be that the number of training iterations reaches a preset threshold. Optionally, it can also be that the loss function value of the initial large-scale video understanding model after training is calculated based on the predicted video understanding information and the annotation information included in the annotated historical videos, and the loss function value reaches a preset loss function value threshold.

[0107] In the process of training an initial large-scale video understanding model based on labeled historical videos, since the video feature understanding layer is a base large-scale language model, its parameters and weights do not need to be adjusted during training. Only the parameters and weights of the video feature extraction layer need to be adjusted, which improves the efficiency of model training. Furthermore, because the base large-scale speech model has strong understanding capabilities, the initial large-scale video understanding model has strong video understanding capabilities, resulting in a strong overall video understanding model after training. This improves the accuracy of determining video understanding information based on the model, and further enhances the accuracy of extreme scene recognition. In the process of developing the large-scale video understanding model, the labeled historical videos used are not limited to those depicting extreme scenes. Therefore, the number of training samples increases, improving the accuracy of model training and the accuracy of the resulting video understanding model, further enhancing the accuracy of determining video understanding information based on the model.

[0108] The video understanding model construction process provided in this embodiment of the invention involves acquiring an initial video understanding model and labeled historical videos. The initial video understanding model is then trained based on the labeled historical videos to obtain the final video understanding model. The initial video understanding model is constructed based on a convolutional neural network and a large language model. Therefore, the video understanding model constructed based on the initial model can accurately extract video feature information from the video to be identified and accurately understand this video feature information, improving the accuracy of the obtained video understanding information and thus enhancing the accuracy of extreme scene recognition. The labeled historical videos are videos of the surrounding environment acquired during the vehicle's historical autonomous driving process, which are easily obtained, thus enriching the training set of the initial video understanding model and further improving the accuracy of the trained video understanding model, thereby enhancing the accuracy of extreme scene recognition.

[0109] Figure 4 This is a schematic diagram of the structure of an extreme scene recognition device provided in an embodiment of the present invention, as shown below. Figure 4As shown, the extreme scene recognition device 400 of this embodiment includes:

[0110] The acquisition module 401 is used to acquire a video of the surrounding environment of the vehicle to be identified during the autonomous driving process of the vehicle.

[0111] Processing module 402 is used to input the video to be recognized into a pre-built video understanding big model for processing to obtain video understanding information of the video to be recognized; wherein, the video understanding big model is a model for understanding and describing the video to be recognized; the video understanding information indicates at least one extreme scene object; the extreme scene object refers to a video object used to identify whether the video includes an extreme scene;

[0112] The recognition module 403 is used to perform extreme scene recognition processing on the video to be recognized based on video understanding information, and to obtain and display the extreme scene recognition result; wherein, the extreme scene recognition result is used to indicate whether the video to be recognized includes extreme scenes.

[0113] In one possible embodiment, the apparatus further includes a construction module for acquiring an initial large-scale video understanding model and labeled historical videos; wherein the initial large-scale video understanding model is an initial model constructed based on a convolutional neural network and a large language model; wherein the labeled historical videos include historical videos and annotation information; the historical videos are videos of the surrounding environment acquired during the vehicle's historical autonomous driving process; the annotation information is historical video understanding information corresponding to the historical information; and the initial large-scale video understanding model is trained based on the labeled historical videos to obtain a large-scale video understanding model.

[0114] In one possible embodiment, the initial video understanding big model includes an initial video feature extraction layer, a cue word generation layer, and a video feature understanding layer; wherein, the initial video feature extraction layer is a network layer constructed based on a convolutional neural network to extract video features; the cue word generation layer is a pre-constructed network layer that converts video feature information into video feature cue words; the video feature understanding layer is a network layer constructed based on a base big language model for generating video understanding information; the construction module is specifically used to input historical videos from labeled historical videos into the initial video feature extraction layer included in the initial video understanding big model to obtain corresponding predicted video feature information; and to process the predicted video feature information according to the cue word generation layer included in the initial video understanding big model. The process involves generating prompts to obtain corresponding historical video feature understanding prompts. These prompts are then input into the video feature understanding layer of the initial video understanding model to obtain corresponding predicted video understanding information. Based on the predicted video understanding information and the annotation information included in the labeled historical videos, the parameters and weights of the initial video feature extraction layer are adjusted to obtain the trained initial video understanding model. If the trained initial video understanding model meets the preset convergence conditions, it is determined as the video understanding model. Otherwise, the process of training the initial video understanding model based on the labeled historical videos is repeated until the trained initial video understanding model meets the preset convergence conditions.

[0115] In one possible embodiment, the recognition module 403 is specifically configured to determine at least one extreme scene object and the relative relationship between each extreme scene object based on video understanding information; determine whether the video to be recognized includes an extreme scene based on the at least one extreme scene object and the relative relationship between each extreme scene object; if it is determined that the video to be recognized includes an extreme scene, then determine the scene type, scene location, scene timestamp, and severity level of the extreme scene, and generate and display the corresponding extreme scene recognition result based on the scene type, scene location, scene timestamp, and severity level of the extreme scene; otherwise, directly generate and display the corresponding extreme scene recognition result.

[0116] In one possible embodiment, the processing module 402 is further configured to determine the environmental information and scene information of the video to be identified; determine the corresponding video preprocessing strategy based on the environmental information and scene information; and preprocess the video to be identified according to the corresponding video preprocessing strategy.

[0117] In one possible embodiment, the extreme scene recognition device further includes an application module for performing autonomous driving optimization processing on the video to be recognized if the extreme scene recognition result indicates that the video to be recognized includes an extreme scene.

[0118] The apparatus of this invention can be used to execute the technical solutions of any of the method embodiments shown above. Its implementation principle and technical effect are similar, and will not be repeated here.

[0119] Figure 5 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present invention. Figure 5 As shown, the electronic device 500 may include at least one processor 501 and a memory 502.

[0120] Memory 502 is used to store programs. Specifically, the program may include program code, which includes computer-executable instructions.

[0121] The memory 502 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0122] The processor 501 executes computer execution instructions stored in the memory 502 to implement the extreme scenario identification method described in the foregoing method embodiments. The processor 501 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. Specifically, when implementing the extreme scenario identification method described in the foregoing method embodiments, the electronic device may be, for example, an electronic control unit in a vehicle, a server, or other electronic device with processing capabilities.

[0123] Optionally, the electronic device 500 may also include a communication interface 503. In specific implementations, if the communication interface 503, memory 502, and processor 501 are implemented independently, they can be interconnected via a bus to complete communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not imply that there is only one bus or one type of bus.

[0124] Optionally, in a specific implementation, if the communication interface 503, memory 502, and processor 501 are integrated on a single chip, then the communication interface 503, memory 502, and processor 501 can communicate through an internal interface.

[0125] The present invention also provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implements the above-mentioned extreme scenario identification method.

[0126] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method for identifying extreme scenarios.

[0127] The present invention also provides an automobile, including an electronic device that implements the above-described method for recognizing extreme scenarios.

[0128] The aforementioned computer-readable storage media can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0129] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in an identification device for extreme scenarios.

[0130] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0131] Finally, it should be noted that the above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A method for identifying extreme scenes, characterized in that: include: During the automatic driving of the vehicle, obtaining a video to be identified of the surrounding environment of the vehicle; The video to be identified is input into a pre-built video understanding model for processing to obtain video understanding information of the video to be identified; wherein the video understanding model is a model for understanding and describing the video to be identified; the video understanding information indicates at least one extreme scene object; the extreme scene object refers to a video object used to identify whether the video includes an extreme scene; According to the video understanding information, the video to be identified is processed for extreme scenes, and an extreme scene identification result is obtained and displayed; wherein the extreme scene identification result is used to indicate whether the video to be identified includes extreme scenes.

2. The method according to claim 1, characterized in that The construction process of the video understanding large model includes: Acquire an initial video understanding large model and annotated historical videos; wherein the initial video understanding large model is an initial model constructed based on a convolutional neural network and a large language model; wherein the annotated historical videos include historical videos and annotated information; the historical videos are videos of the surrounding environment acquired during the historical automatic driving process of the vehicle; the annotated information is historical video understanding information corresponding to the historical information; The initial video understanding large model is trained according to the labeled historical videos to obtain the video understanding large model.

3. The method according to claim 2, characterized in that The initial video understanding model includes an initial video feature extraction layer, a prompt word generation layer, and a video feature understanding layer; wherein the initial video feature extraction layer is a network layer constructed based on a convolutional neural network to extract video features; the prompt word generation layer is a pre-constructed network layer that converts video feature information into video feature prompt words; the video feature understanding layer is a network layer constructed based on a base large language model for generating video understanding information; The initial video understanding model is trained according to the annotated historical videos to obtain the video understanding model, including: Inputting the historical videos in the annotated historical videos into the initial video feature extraction layer included in the initial video understanding large model to obtain corresponding predicted video feature information; According to the prompt word generation layer included in the initial video understanding large model, the predicted video feature information is processed by prompt word generation to obtain corresponding historical video feature understanding prompt words; and the historical video feature understanding prompt words are input into the video feature understanding layer included in the initial video understanding large model to obtain corresponding predicted video understanding information; According to the predicted video understanding information and the annotation information included in the annotated historical video, the parameters and weights of the initial video feature extraction layer are adjusted to obtain a trained initial video understanding large model; If the trained initial video understanding big model meets the preset convergence conditions, the trained initial video understanding big model is determined as the video understanding big model; otherwise, the process of training the initial video understanding big model according to the labeled historical video is repeated until the trained initial video understanding big model meets the preset convergence conditions.

4. The method according to claim 1, characterized in that: According to the video understanding information, the video to be identified is subjected to extreme scene identification processing, and an extreme scene identification result is obtained and displayed, including: Determining at least one of the extreme scene objects and a relative relationship between the extreme scene objects according to the video understanding information; Determining whether the to-be-identified video includes an extreme scene according to the at least one extreme scene object and the relative relationship between the extreme scene objects; If it is determined that the video to be identified includes extreme scenes, the scene type, scene location, scene timestamp, and severity level of the extreme scene are determined, and based on the scene type, scene location, scene timestamp, and severity level of the extreme scene, the corresponding extreme scene identification result is generated and displayed; otherwise, the corresponding extreme scene identification result is directly generated and displayed.

5. The method according to any one of claims 1 to 4, characterized in that: After obtaining the video to be identified, the method further includes: Determine the environment information and scene information of the video to be identified; Determining a corresponding video preprocessing strategy according to the environment information and the scene information; The video to be identified is preprocessed according to a corresponding video preprocessing strategy.

6. The method according to claim 5, characterized in that The method further comprises: If the extreme scene recognition result indicates that the video to be recognized includes an extreme scene, automatic driving optimization processing of the vehicle is performed based on the video to be recognized.

7. An extreme scene recognition device, characterized in that: include: An acquisition module, used to acquire a video to be identified of the surrounding environment of the vehicle during the automatic driving of the vehicle; A processing module is used to input the video to be identified into a pre-built video understanding model for processing, and obtain video understanding information of the video to be identified; wherein the video understanding model is a model for understanding and describing the video to be identified; the video understanding information indicates at least one extreme scene object; the extreme scene object refers to a video object used to identify whether the video includes an extreme scene; The recognition module is used to perform extreme scene recognition processing on the video to be recognized based on the video understanding information, and obtain and display extreme scene recognition results; wherein the extreme scene recognition results are used to indicate whether the video to be recognized includes extreme scenes.

8. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

10. An automobile, characterized in that: An electronic device comprising executing the extreme scenario recognition method as described in any one of claims 1-6.

Images