Environmental detection method, device, electronic equipment and computer storage medium

Abstract

The present application provides an environment detection method, device, electronic device and computer storage medium, receiving a video to be detected; for each frame of the video to be detected, the image of the frame is input into an environment detection model, and the environment detection result of the frame is output, the environment detection model includes a feature extraction module, a pyramid bottom-up feature network, a target detection head, a vehicle state detection head and a key point detection head; the environment detection result includes the target feature and grounding point information of each object in the image; the target feature of the object includes category information and location information; if the category of the object is a vehicle, the environment detection result also includes the vehicle state information output by the vehicle state detection head, and the grounding point information output by the key point detection head includes the grounding point position and grounding point state information of the vehicle; if the object is not a vehicle, the grounding point information output by the key point detection head includes the grounding point position. Provide detailed environment perception information for the automatic parking system quickly and effectively.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an environment detection method, device, electronic equipment and computer storage medium. Background Art

[0002] In the intelligent parking technology of autonomous driving, environmental perception is the first link. Its core is to enable the vehicle to pay attention to the situation around the vehicle at any time and respond in time during the process of searching for a parking garage and parking in and out. In practical applications, it is necessary not only to identify the vehicle from multiple angles at the same time, but also to consider the identification of many actual targets, including vehicles, pedestrians, pillars and other types of targets, to provide a basis for subsequent path planning, real-time decision-making and driving control.

[0003] Considering the safety of automatic parking, the perception algorithm has very high requirements for accuracy and efficiency. In the existing technology, the surround perception algorithm based on deep learning is difficult to deploy based on the above requirements. At the same time, due to the complex and changeable nature of the actual situation, there are also very high requirements for the number of categories supported by the surround perception algorithm. Summary of the invention

[0004] In view of this, the present application provides an environment detection method, device, electronic device and computer storage medium to quickly and effectively provide detailed environment perception information for an automatic parking system.

[0005] The first aspect of the present application provides a method for detecting an environment, comprising:

[0006] Receive the video to be detected;

[0007] For each frame of the video to be detected, the image of the frame is input into the environment detection model, and the environment detection result of the frame is output; wherein the environment detection model includes a feature extraction module, a pyramid bottom-up feature network and three detection branches; the three detection branches are a target detection head, a vehicle state detection head and a key point detection head respectively; the environment detection result includes the target features and grounding point information of each object in the image; the target features of the object include category information and position information; if the category of the object is a vehicle, the environment detection result also includes the vehicle state information output by the vehicle state detection head, and the grounding point information output by the key point detection head includes the grounding point position and grounding point state information of the vehicle; if the object is not a vehicle, the grounding point information output by the key point detection head includes the grounding point position.

[0008] Optionally, for each frame of the video to be detected, inputting the picture of the frame into the environment detection model, and before outputting the environment detection result, the method further includes:

[0009] For each frame of the video to be detected, the image of the frame is processed according to a preset enhancement method to obtain a preprocessed image.

[0010] Optionally, after the environment detection model receives the picture, the feature extraction module performs feature extraction on the picture to obtain multiple feature maps of different sizes; uses the pyramid bottom-up feature network to aggregate all the feature maps to obtain aggregate features of each object in the picture, and inputs the aggregate features of the object into the target detection head to obtain the target features of the object; if the category of the object is a vehicle, the aggregate features of the object are input into the vehicle status detection head, and the vehicle status information is output; the aggregate features of the object are input into the key point detection head, and the grounding point information including the grounding point position and grounding point status information of the vehicle is output; if the object is not a vehicle, the aggregate features of the object are input into the key point detection head, and the grounding point information including the grounding point position is output.

[0011] Optionally, the feature extraction module is a backbone feature extraction network of YoloV7.

[0012] A second aspect of the present application provides an environment detection device, comprising:

[0013] A receiving unit, used for receiving a video to be detected;

[0014] The environment detection unit is used to input the image of each frame of the video to be detected into the environment detection model, and output the environment detection result of the frame; wherein the environment detection model includes a feature extraction module, a pyramid bottom-up feature network and three detection branches; the three detection branches are a target detection head, a vehicle state detection head and a key point detection head respectively; the environment detection result includes the target features and grounding point information of each object in the image; the target features of the object include category information and position information; if the category of the object is a vehicle, the environment detection result also includes the vehicle state information output by the vehicle state detection head, and the grounding point information output by the key point detection head includes the grounding point position and grounding point state information of the vehicle; if the object is not a vehicle, the grounding point information output by the key point detection head includes the grounding point position.

[0015] Optionally, the environment detection device further includes:

[0016] The preprocessing unit is used to process the image of each frame of the video to be detected according to a preset enhancement method to obtain a preprocessed image.

[0017] Optionally, after the environment detection model receives the picture, the feature extraction module performs feature extraction on the picture to obtain multiple feature maps of different sizes; uses the pyramid bottom-up feature network to aggregate all the feature maps to obtain aggregate features of each object in the picture, and inputs the aggregate features of the object into the target detection head to obtain the target features of the object; if the category of the object is a vehicle, the aggregate features of the object are input into the vehicle status detection head, and the vehicle status information is output; the aggregate features of the object are input into the key point detection head, and the grounding point information including the grounding point position and grounding point status information of the vehicle is output; if the object is not a vehicle, the aggregate features of the object are input into the key point detection head, and the grounding point information including the grounding point position is output.

[0018] Optionally, the feature extraction module is a backbone feature extraction network of YoloV7.

[0019] A third aspect of the present application provides an electronic device, including:

[0020] one or more processors;

[0021] a storage device having one or more programs stored thereon;

[0022] When the one or more programs are executed by the one or more processors, the one or more processors implement the environment detection method as described in any one of the first aspects.

[0023] A fourth aspect of the present application provides a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for detecting an environment as described in any one of the first aspects is implemented.

[0024] As can be seen from the above scheme, the present application provides an environment detection method, device, electronic device and computer storage medium, receiving a video to be detected; for each frame of the video to be detected, the image of the frame is input into the environment detection model, and the environment detection result of the frame is output; wherein the environment detection model includes a feature extraction module, a pyramid bottom-up feature network and three detection branches; the three detection branches are a target detection head, a vehicle state detection head and a key point detection head; the environment detection result includes the target features and grounding point information of each object in the image; the target features of the object include category information and location information; if the category of the object is a vehicle, the environment detection result also includes the vehicle state information output by the vehicle state detection head, and the grounding point information output by the key point detection head includes the grounding point position and grounding point state information of the vehicle; if the object is not a vehicle, the grounding point information output by the key point detection head includes the grounding point position. Provide detailed environment perception information for the automatic parking system quickly and effectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0026] Figure 1 A specific flow chart of a method for detecting an environment provided in an embodiment of the present application;

[0027] Figure 2 A schematic diagram of a network structure of an environment detection model provided in another embodiment of the present application;

[0028] Figure 3 A schematic diagram of an environment detection device provided by another embodiment of the present application;

[0029] Figure 4 A schematic diagram of an electronic device for implementing an environment detection method provided in another embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0032] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0033] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0034] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0035] The present application embodiment provides a method for detecting an environment, such as Figure 1 As shown, the specific steps include:

[0036] S101: Receive a video to be detected.

[0037] S102: For each frame of the video to be detected, input the image of the frame into the environment detection model, and output the environment detection result of the frame.

[0038] Among them, the environment detection model includes a feature extraction module, a pyramid bottom-up feature network and three detection branches; the three detection branches are respectively a target detection head, a vehicle state detection head and a key point detection head; the environment detection result includes the target features and grounding point information of each object in the picture; the target features of the object include category information and position information; if the category of the object is a vehicle, the environment detection result also includes the vehicle state information output by the vehicle state detection head, and the grounding point information output by the key point detection head includes the grounding point position and grounding point state information of the vehicle; if the object is not a vehicle, the grounding point information output by the key point detection head includes the grounding point position.

[0039] It should be noted that vehicles include but are not limited to cars, trucks, buses and construction vehicles, and non-vehicles include but are not limited to people, pillars, railings, etc., which are not limited here.

[0040] Vehicle status information includes but is not limited to Normal: vehicles that are not blocked or are slightly blocked, Truncated: vehicles that are located at the edge of the entire image and are truncated, Crowded: vehicles blocked by targets, etc. The grounding point status information is subdivided into: Visible point: grounding point visible to the tire, Prediction point: grounding point invisible to the tire, the location of the tire grounding point that needs to be predicted, etc., which are not limited here.

[0041] Optionally, in another embodiment of the present application, an implementation of the environment detection method includes, for each frame of the video to be detected, inputting the image of the frame into the environment detection model, and outputting the environment detection result before further including:

[0042] For each frame of the video to be detected, the image of the frame is processed according to a preset enhancement method to obtain a preprocessed image.

[0043] The preset enhancement methods include, but are not limited to, a series of enhancement operations such as scaling, flipping, color change, MOSiAC, MIXUP, etc. on the image, which are not limited here.

[0044] Optionally, in another embodiment of the present application, an implementation of a method for training an environment detection model includes:

[0045] (1) Obtaining raw data and labels

[0046] This application collects a total of 150,000 data in different scenarios in multiple cities, uses annotation tools for manual annotation, and obtains image annotation results. Annotation is performed in the form of a closed rectangle, and the target box strictly fits the target body. At the same time, the grounding point of non-vehicle targets is marked with a solid dot; the vehicle status information of vehicle targets includes "normal", "truncated", and "occluded", and the vehicle grounding point position is marked with a solid dot, and the vehicle tire grounding point and grounding point status information include "visible" and "invisible". Vehicle targets are divided into cars, buses, trucks, and engineering vehicles, which are not limited here.

[0047] (2) Construction of network structure

[0048] This application uses the YoloV7 algorithm of the One_stage series as the core, builds the framework of the target detection head, vehicle state detection head and key point detection head, generates a detection priori box based on the existing K-means clustering algorithm and the actual target size, extracts features through the YoloV7 backbone, and outputs feature maps of three different sizes. At the same time, in order to solve the problem of small target detection, this application introduces MOSiAC and MIXUP for data enhancement, and fuses the large, medium and small size feature maps through the pyramid bottom-up feature network (PAFPN) structure to obtain aggregated features, so as to achieve the purpose of detecting the features of the target object.

[0049] The entire network structure is divided into three branches, including: target detection head, vehicle state detection head and key point detection head. Specifically, the fusion features are input into the target detection head, vehicle state detection head and key point detection head to obtain the environmental detection results; each branch is connected to multiple corresponding fully connected layers.

[0050] It should be noted that in order to measure the distance to the target later, this application introduces KeypointHead to predict the target grounding point, and introduces CarStateHead to judge the vehicle state and select the appropriate view considering that the vehicle will appear in different perspectives at the same time. The multi-Head training method is adopted to obtain the grounding point coordinates of non-vehicle targets and the state information of vehicle targets at the same time, as well as the grounding point coordinates and state information, which reduces the resource consumption and repeated calculations when using multiple models, and effectively improves the calculation speed.

[0051] (3) Model training

[0052] Based on the Pytorch deep learning training framework, using distributed training, each branch calculates the difference between the prediction result and the label, position, and ground point label of the target matching box of the corresponding training sample, and back-propagates to complete the parameter update of the corresponding task head. The difference of each task head is added and back-propagated layer by layer to the multi-task parallel convolutional neural network to complete the overall parameter update and iterative training of the model. Among them, the cross entropy is used as the loss function for the classifier and confidence of the detection head, and CIoU is used as the loss function for the bounding box:

[0053]

[0054]

[0055]

[0056] in, is the loss function, is the area of ​​the intersection of the two bounding boxes divided by the area of ​​their union, is the center point of the two bounding boxes, is the Euclidean distance between two bounding boxes, represents the distance between the diagonals of the enclosing areas of the two bounding boxes, is the weight coefficient, is a measure of the consistency of the relative proportions of the two bounding boxes. , , and are the width and height of the two bounding boxes, is the inverse tangent function.

[0057] The vehicle state head classifier also uses cross entropy as the loss function; the key point head classifier also uses cross entropy as the loss function, and the ground point regression uses the SmoothL1 function as the loss function. The training of the detection and recognition modules is carried out simultaneously, which greatly improves the training speed of the model. The following is the formula of the SmoothL1 loss function, where x is the input tensor:

[0058]

[0059] In the actual application process of this application, if Figure 2As shown in the figure, after the environment detection model (model) receives the image (processed image), the feature extraction module (YOLOv7 backbone) extracts features from the image to obtain multiple feature maps of different sizes (large feature map, medium feature map, small feature map, where large feature map, medium feature map and small feature map are obtained by downsampling the original image by 8 times, 16 times and 32 times, respectively. It can be understood that this is just an example); all feature maps are aggregated using the pyramid bottom-up feature network (PAFANncek) to obtain the aggregated features of each object in the image, and the aggregated features of the object are input into the target detection head (YOLOv7 head) to obtain the category information (object class) and location information (objectbounding box) of the object. If the category of the object is a vehicle, the aggregated features of the object are input into the vehicle state detection head (carstate head), and the vehicle state information (object state attributes) is output. The aggregated features of the object are input into the key point detection head (object keypoints head), and the ground point information (object The keypoints include the vehicle's grounding point location and grounding point status information; if the object is not a vehicle, the aggregated features of the object are input to the keypoint detection head (object keypoints head), and the grounding point information (object keypoints) including the grounding point location is output. Parallel branches are added on the basis of the detection model, basic features are shared, and reasoning is completed by a single model. The computational complexity is low, which greatly reduces the time consumption of the entire process and is applicable to the deployment of any vehicle.

[0060] Optionally, in the specific implementation process of the present application, the output results of the target detection head, the vehicle state detection head and the key point detection head may be spliced ​​to obtain the final output result.

[0061] As can be seen from the above scheme, the present application provides a method for detecting an environment. After receiving the video to be detected, for each frame of the video to be detected, the image of the frame is input into the environment detection model, and the environment detection result of the frame is output; wherein the environment detection model includes a feature extraction module, a pyramid bottom-up feature network and three detection branches; the three detection branches are a target detection head, a vehicle state detection head and a key point detection head; the environment detection result includes the target features and grounding point information of each object in the image; the target features of the object include category information and location information; if the category of the object is a vehicle, the environment detection result also includes the vehicle state information output by the vehicle state detection head, and the grounding point information output by the key point detection head includes the grounding point position and grounding point state information of the vehicle; if the object is not a vehicle, the grounding point information output by the key point detection head includes the grounding point position. Provide detailed environmental perception information for the automatic parking system quickly and effectively.

[0062] The present application embodiment provides an environment detection device, such as Figure 3 As shown, specifically including:

[0063] The receiving unit 301 is used to receive a video to be detected.

[0064] The environment detection unit 302 is used to input the image of each frame of the video to be detected into the environment detection model, and output the environment detection result of the frame.

[0065] Among them, the environment detection model includes a feature extraction module, a pyramid bottom-up feature network and three detection branches; the three detection branches are respectively a target detection head, a vehicle state detection head and a key point detection head; the environment detection result includes the target features and grounding point information of each object in the picture; the target features of the object include category information and position information; if the category of the object is a vehicle, the environment detection result also includes the vehicle state information output by the vehicle state detection head, and the grounding point information output by the key point detection head includes the grounding point position and grounding point state information of the vehicle; if the object is not a vehicle, the grounding point information output by the key point detection head includes the grounding point position.

[0066] For the specific working process of the units disclosed in the above embodiments of the present application, please refer to the corresponding method embodiments, such as Figure 1 As shown, no further description is given here.

[0067] Optionally, in another embodiment of the present application, an implementation of the environment detection device further includes:

[0068] The preprocessing unit is used to process the image of each frame of the video to be detected according to a preset enhancement method to obtain a preprocessed image.

[0069] The specific working process of the units disclosed in the above embodiments of the present application can be found in the corresponding method embodiments, which will not be repeated here.

[0070] Optionally, in another embodiment of the present application, after the environment detection model receives the image, the feature extraction module extracts features from the image to obtain multiple feature maps of different sizes; all feature maps are aggregated using a pyramid bottom-up feature network to obtain aggregate features of each object in the image, and the aggregate features of the object are input into the target detection head to obtain the target features of the object; if the object is a vehicle, the aggregate features of the object are input into the vehicle status detection head, and the vehicle status information is output; the aggregate features of the object are input into the key point detection head, and the grounding point information including the grounding point position and grounding point status information of the vehicle is output; if the object is not a vehicle, the aggregate features of the object are input into the key point detection head, and the grounding point information including the grounding point position is output.

[0071] The specific working process of the units disclosed in the above embodiments of the present application can be found in the corresponding method embodiments, which will not be repeated here.

[0072] Optionally, in another embodiment of the present application, the feature extraction module is a backbone feature extraction network of YoloV7.

[0073] The specific working process of the units disclosed in the above embodiments of the present application can be found in the corresponding method embodiments, which will not be repeated here.

[0074] As can be seen from the above scheme, the present application provides an environment detection device, wherein the receiving unit 301 receives the video to be detected; the environment detection unit 302 inputs the picture of each frame of the video to be detected into the environment detection model, and outputs the environment detection result of the frame; wherein the environment detection model includes a feature extraction module, a pyramid bottom-up feature network and three detection branches; the three detection branches are a target detection head, a vehicle state detection head and a key point detection head; the environment detection result includes the target features and grounding point information of each object in the picture; the target features of the object include category information and location information; if the category of the object is a vehicle, the environment detection result also includes the vehicle state information output by the vehicle state detection head, and the grounding point information output by the key point detection head includes the grounding point position and grounding point state information of the vehicle; if the object is not a vehicle, the grounding point information output by the key point detection head includes the grounding point position. Provide detailed environment perception information for the automatic parking system quickly and effectively.

[0075] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0076] Another embodiment of the present application provides an electronic device, such as Figure 4 As shown, including:

[0077] One or more processors 401 .

[0078] The storage device 402 stores one or more programs.

[0079] When the one or more programs are executed by the one or more processors 401, the one or more processors 401 implement the environment detection method as described in any one of the above embodiments.

[0080] Another embodiment of the present application provides a computer storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, the method for detecting an environment as described in any one of the above embodiments is implemented.

[0081] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0082] It should be noted that the computer-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0083] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0084] Another embodiment of the present application provides a computer program product. When the computer program product is executed, it is used to execute the above-mentioned environment detection method.

[0085] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, the above-mentioned functions defined in the method of the embodiment of the present application are executed.

[0086] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in this application is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing this application.

[0087] Although several specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present application. Certain features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination.

[0088] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of application involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above application concept. For example, the above features are replaced with (but not limited to) technical features with similar functions applied in the present application.

Claims

1. A method for detecting an environment, characterized in that: include: Receive the video to be detected; For each frame of the video to be detected, the image of the frame is input into the environment detection model, and the environment detection result of the frame is output; wherein the environment detection model includes a feature extraction module, a pyramid bottom-up feature network and three detection branches; the three detection branches are a target detection head, a vehicle state detection head and a key point detection head; the environment detection result includes the target features and grounding point information of each object in the image; the target features of the object include category information and position information; if the category of the object is a vehicle, the environment detection result also includes the vehicle state information output by the vehicle state detection head, and the grounding point information output by the key point detection head includes the grounding point position and grounding point state information of the vehicle; If the object is not a vehicle, the grounding point information output by the key point detection head includes the grounding point position.

2. The method for detecting an environment according to claim 1, characterized in that: For each frame of the video to be detected, the image of the frame is input into the environment detection model, and before the environment detection result is output, the method further includes: For each frame of the video to be detected, the image of the frame is processed according to a preset enhancement method to obtain a preprocessed image.

3. The method for detecting an environment according to claim 1, characterized in that: After the environment detection model receives the picture, the feature extraction module extracts features from the picture to obtain a plurality of feature maps of different sizes; the pyramid bottom-up feature network is used to aggregate all the feature maps to obtain aggregate features of each object in the picture, and the aggregate features of the object are input into the target detection head to obtain the target features of the object; if the category of the object is a vehicle, the aggregate features of the object are input into the vehicle state detection head to output the vehicle state information; the aggregate features of the object are input into the key point detection head to output the grounding point information including the grounding point position and grounding point state information of the vehicle; If the object is not a vehicle, the aggregated features of the object are input into a key point detection head, and the grounding point information including the grounding point position is output.

4. The method for detecting an environment according to claim 1, characterized in that: The feature extraction module is the backbone feature extraction network of YoloV7.

5. An environment detection device, characterized in that: include: A receiving unit, used for receiving a video to be detected; The environment detection unit is used for inputting the picture of each frame of the video to be detected into the environment detection model, and outputting the environment detection result of the frame; wherein the environment detection model includes a feature extraction module, a pyramid bottom-up feature network and three detection branches; the three detection branches are a target detection head, a vehicle state detection head and a key point detection head; the environment detection result includes the target feature and grounding point information of each object in the picture; the target feature of the object includes category information and position information; if the category of the object is a vehicle, the environment detection result also includes the vehicle state information output by the vehicle state detection head, and the grounding point information output by the key point detection head includes the grounding point position and grounding point state information of the vehicle; If the object is not a vehicle, the grounding point information output by the key point detection head includes the grounding point position.

6. The environment detection device according to claim 5, characterized in that: Also includes: The preprocessing unit is used to process the image of each frame of the video to be detected according to a preset enhancement method to obtain a preprocessed image.

7. The environment detection device according to claim 5, characterized in that: After the environment detection model receives the picture, the feature extraction module extracts features from the picture to obtain a plurality of feature maps of different sizes; the pyramid bottom-up feature network is used to aggregate all the feature maps to obtain aggregate features of each object in the picture, and the aggregate features of the object are input into the target detection head to obtain the target features of the object; if the category of the object is a vehicle, the aggregate features of the object are input into the vehicle state detection head to output the vehicle state information; the aggregate features of the object are input into the key point detection head to output the grounding point information including the grounding point position and grounding point state information of the vehicle; If the object is not a vehicle, the aggregated features of the object are input into a key point detection head, and the grounding point information including the grounding point position is output.

8. The environment detection device according to claim 5, characterized in that: The feature extraction module is the backbone feature extraction network of YoloV7.

9. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the environment detection method as claimed in any one of claims 1 to 4.

10. A computer storage medium, characterized in that: A computer program is stored thereon, wherein when the computer program is executed by a processor, the method for detecting an environment as claimed in any one of claims 1 to 4 is implemented.

Images