A window control method, system, and vehicle

Abstract

An embodiment of the present application provides a window control method, system, and vehicle. The method includes: obtaining an environmental image around the vehicle; inputting the environmental image into an environmental detection model to obtain environmental information outside the vehicle window, where the environmental detection model is trained based on multiple sample environmental images with environmental labels outside the vehicle window; and in response to the environmental information, executing a window control strategy corresponding to the environmental information. The window control method provided by the present application determines the environmental information outside the vehicle window by detecting the environmental image around the vehicle and executes a window control strategy corresponding to the environmental information, without the need for the driver to manually control the window button to control the opening and closing of the window. On the one hand, it avoids the problem of the driver being distracted while driving and causing traffic accidents; on the other hand, it can sense the environmental information around the vehicle and control the window according to the environmental information. This improves the driving safety and experience of the driver.

Description

Technical Field

[0001] The present application relates to the technical field of vehicle control, and particularly to a window control method, system and vehicle. Background Art

[0002] With the continuous development of automotive technology, driving safety and driving experience have attracted more and more attention. Especially when a vehicle is driving on the road, it often encounters situations where the vehicle picks up dust when passing through a gravel road surface, or when driving on a section with debris or smoke. If the vehicle's windows are in the open state, these dust, debris or smoke can easily enter the vehicle, bringing great troubles to the drivers and passengers in the vehicle.

[0003] In the prior art, when the driver recognizes the existence of dust, debris or smoke, etc., the driver needs to manually control the window button to control the window. On the one hand, it will cause the driver to be distracted and lead to traffic accidents; on the other hand, when the driver recognizes the dust, debris or smoke, the dust, debris or smoke has already entered the vehicle, affecting the driving safety and experience of the driver.

[0004] To solve the above problems, the present application proposes a window control method. Summary of the Invention

[0005] In view of the above problems, the embodiments of the present application provide a window control method, system and vehicle to solve the problems in the prior art that when the driver recognizes the existence of dust, debris or smoke, etc. around the vehicle, the driver needs to manually control the window button to control the opening and closing of the window. On the one hand, it will cause the driver to be distracted and lead to traffic accidents; on the other hand, when the driver recognizes the dust, debris or smoke, the dust, debris or smoke has already entered the vehicle, thus affecting the driving safety and experience of the driver.

[0006] In the first aspect of the embodiments of the present application, a window control method is provided, and the method includes:

[0007] Obtain an environmental image around the vehicle;

[0008] Input the environmental image into an environmental detection model to obtain environmental information outside the vehicle window, and the environmental detection model is trained based on a plurality of sample environmental images with environmental labels outside the vehicle window;

[0009] Respond to the environmental information and execute a window control strategy corresponding to the environmental information.

[0010] Optionally, the step of inputting the environmental image into the environmental detection model to obtain environmental information outside the vehicle window includes:

[0011] Input the environmental image into the environmental detection model to identify whether at least one target object in the environmental image is at least one of sundries, soot, or smoke;

[0012] If the at least one target object is at least one of the sundries, the soot, or the smoke, determine that the environmental information is abnormal environmental information;

[0013] If the at least one target object is not any of the sundries, the soot, or the smoke, determine that the environmental information is normal environmental information.

[0014] Optionally, inputting the environmental image into the environmental detection model to obtain environmental information outside the vehicle window includes:

[0015] Input the environmental image into the environmental detection model to identify weather information in the environmental image.

[0016] Optionally, in response to the environmental information, executing a window control strategy corresponding to the environmental information includes:

[0017] When the weather information represents abnormal weather, control the vehicle's voice system to emit a first voice, which is used to prompt the driver that the weather is abnormal and ask whether to choose to close the window;

[0018] When the weather information represents normal weather, control the vehicle's voice system to emit a second voice, which is used to prompt the driver whether to choose to open the window or whether to choose to turn on the air conditioner;

[0019] In response to the detected feedback instruction of the driver for the first voice or the second voice, execute a window control strategy corresponding to the feedback instruction.

[0020] Optionally, in response to the environmental information, executing a window control strategy corresponding to the environmental information includes:

[0021] When the environmental information is the abnormal environmental information and the window is in the open state, control the window to close;

[0022] When the environmental information is the normal environmental information and the window is in the closed state, control the window to open.

[0023] Optionally, in response to the environmental information, executing a window control strategy corresponding to the environmental information includes:

[0024] When the weather information represents abnormal weather and the window is in the open state, control the window to close;

[0025] When the weather information represents normal weather and the vehicle window is in a closed state, control the vehicle window to open.

[0026] Optionally, in the process of executing the vehicle window control strategy corresponding to the environmental information, it further includes:

[0027] Determine whether a control instruction of the driver for the vehicle window is detected;

[0028] When the control instruction of the driver for the vehicle window is detected, stop executing the vehicle window control strategy corresponding to the environmental information;

[0029] In response to the control instruction of the driver, execute the vehicle window control strategy corresponding to the control instruction of the driver.

[0030] Optionally, the environmental detection model is obtained by inputting a plurality of sample environmental images with outdoor environmental labels into the YOLOv4 network for training.

[0031] Optionally, the environmental detection model uses Mobilenetv2 as the backbone network to reduce the number of parameters and the amount of calculation of the environmental detection model, and / or,

[0032] The Resunit block of the environmental detection model is a module with an ECA attention mechanism, which is used to improve the attention of the environmental detection model to the target objects in the environmental image.

[0033] In the second aspect of the embodiments of the present application, a vehicle window control system is provided, and the system includes:

[0034] An acquisition module, configured to acquire environmental images around the vehicle;

[0035] An input module, configured to input the environmental image into an environmental detection model to obtain environmental information outside the vehicle window, and the environmental detection model is obtained by training based on a plurality of sample environmental images with outdoor environmental labels;

[0036] An execution module, configured to execute a vehicle window control strategy corresponding to the environmental information in response to the environmental information.

[0037] Optionally, for inputting the environmental image into the environmental detection model to obtain environmental information outside the vehicle window, the input module includes:

[0038] A first input sub-module, configured to input the environmental image into the environmental detection model to identify whether at least one target object in the environmental image is at least one of sundries, soot or smoke;

[0039] A first determination sub-module, configured to determine that the environmental information is abnormal environmental information if at least one of the at least one target object is at least one of the sundries, the soot, or the smoke;

[0040] A second determination sub-module, configured to determine that the environmental information is normal environmental information if at least one of the at least one target object is not any of the sundries, the soot, or the smoke.

[0041] Optionally, for inputting the environmental image into an environmental detection model to obtain environmental information outside the vehicle window, the input module includes:

[0042] A second input sub-module, configured to input the environmental image into the environmental detection model to identify weather information in the environmental image.

[0043] Optionally, for executing a window control strategy corresponding to the environmental information in response to the environmental information, the execution module includes:

[0044] A first control sub-module, configured to control a voice system of the vehicle to emit a first voice for prompting a driver that the weather is abnormal and whether to choose to close the window when the weather information represents abnormal weather;

[0045] A second control sub-module, configured to control the voice system of the vehicle to emit a second voice for prompting the driver whether to choose to open the window or whether to choose to turn on the air conditioner when the weather information represents normal weather;

[0046] A first execution sub-module, configured to execute a window control strategy corresponding to the feedback instruction in response to the detected feedback instruction of the driver for the first voice or the second voice.

[0047] Optionally, for executing a window control strategy corresponding to the environmental information in response to the environmental information, the execution module includes:

[0048] A third control sub-module, configured to control the window to close when the environmental information is the abnormal environmental information and the window is in an open state;

[0049] A fourth control sub-module, configured to control the window to open when the environmental information is the normal environmental information and the window is in a closed state.

[0050] Optionally, for executing a window control strategy corresponding to the environmental information in response to the environmental information, the execution module includes:

[0051] A fifth control sub-module, configured to control the window to close when the weather information indicates abnormal weather and the window is in an open state;

[0052] A sixth control sub-module, configured to control the window to open when the weather information indicates normal weather and the window is in a closed state.

[0053] Optionally, during the execution of the window control strategy corresponding to the environmental information, it further includes:

[0054] A third determination sub-module, configured to determine whether a control instruction for the window is detected by the driver;

[0055] A monitoring sub-module, configured to stop executing the window control strategy corresponding to the environmental information when the control instruction for the window is detected by the driver;

[0056] A second execution sub-module, configured to execute the window control strategy corresponding to the driver's control instruction in response to the driver's control instruction.

[0057] Optionally, the environmental detection model is obtained by inputting the multiple sample environmental images with outdoor environmental labels into the YOLOv4 network for training.

[0058] Optionally, the environmental detection model uses Mobilenetv2 as the backbone network to reduce the number of parameters and computational complexity of the environmental detection model, and / or,

[0059] The Resunit block of the environmental detection model is a module with an ECA attention mechanism, configured to improve the attention of the environmental detection model to the target objects in the environmental image.

[0060] In a third aspect of the embodiments of the present application, a vehicle is provided, and the vehicle includes the window control system as described in the second aspect of the embodiments of the present application.

[0061] The present application has the following advantages:

[0062] An embodiment of the present application provides a window control method, and the method includes: obtaining an environmental image around the vehicle; inputting the environmental image into an environmental detection model to obtain environmental information outside the vehicle window, where the environmental detection model is trained based on a plurality of sample environmental images with environmental labels outside the vehicle window; and in response to the environmental information, executing a window control strategy corresponding to the environmental information. The window control method provided by the present application determines the environmental information outside the vehicle window by detecting the environmental image around the vehicle and executes a window control strategy corresponding to the environmental information, without the need for the driver to manually control the window button to control the opening and closing of the window. On the one hand, it avoids the problem of the driver being distracted while driving and causing traffic accidents; on the other hand, it can sense the environmental information around the vehicle and control the window according to the environmental information. It improves the driving safety and experience of the driver. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0064] Figure 1 is a flowchart of the steps of a window control method provided by an embodiment of the present application;

[0065] Figure 2 is a flowchart of a window control method provided by an embodiment of the present application;

[0066] Figure 3 is a schematic diagram of the network structure of an environmental detection model framework YOLOv4 provided by an embodiment of the present application;

[0067] Figure 4 is a schematic diagram of an ECA attention mechanism provided by an embodiment of the present application;

[0068] Figure 5 is a schematic diagram of the network structure of adding an ECA attention mechanism in a Resunit block provided by an embodiment of the present application;

[0069] Figure 6 is a schematic diagram of a window control system provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0070] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings in the embodiments of the present application. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present application can be more thoroughly understood and the scope of the present application can be fully communicated to those skilled in the art.

[0071] In the prior art, the control of the vehicle window mostly relies on the driver or the passenger. However, when the driver is driving the vehicle and encounters a harsh environment such as dust, debris or smoke around the vehicle, if the driver actively closes the window, it will cause the driver to be distracted, thus triggering a traffic accident. In addition, when the driver recognizes the soot, debris or smoke, the soot, debris or smoke may have already entered the vehicle. The above problems will affect the driving safety and experience of the driver.

[0072] In the first aspect of the embodiments of the present application, a window control method is provided. Referring to Figure 1 , which is a step flowchart of a window control method provided by the embodiments of the present application, the method includes:

[0073] Step S101, obtaining an environmental image around the vehicle;

[0074] Step S102, inputting the environmental image into an environmental detection model to obtain environmental information outside the vehicle window, where the environmental detection model is trained based on a plurality of sample environmental images with environmental labels outside the vehicle window;

[0075] Step S103, in response to the environmental information, executing a window control strategy corresponding to the environmental information.

[0076] In the embodiments of the present application, the method of obtaining the environmental image around the vehicle can be implemented by a camera installed in front of or around the vehicle. Specifically, cameras can be installed at positions such as the front, side, and rear of the vehicle, and the vehicle's head unit system can be used to control the on / off of the cameras. When the vehicle is powered on, the head unit system can automatically detect the status of the cameras and control the cameras to turn on; after the cameras are turned on, the environmental images around the vehicle start to be collected. In practical applications, the camera can use the vehicle's built-in driving recorder to collect environmental images; in order to obtain high-quality image data, high-definition cameras can also be used to collect environmental images, and the present application does not make any limitations here.

[0077] Further, after the camera captures the environmental image, the image data of the environmental image will be sent to the vehicle's in-vehicle system. In practical applications, the image data of the environmental image will be processed within the in-vehicle system. The processing methods include, but are not limited to, converting the image data into a digital format and preprocessing it to improve the accuracy of environmental detection. The preprocessing can include operations such as cropping, scaling, normalization, and color space conversion to ensure that the images input into the environmental detection model have the same size and format and have an identifiable color and brightness range, etc.

[0078] Further, the in-vehicle system of the vehicle inputs the processed environmental image into a pre-trained environmental detection model to detect the environmental information in the environmental image through the environmental detection model.

[0079] In practical applications, it is necessary to pre-train an environmental detection model to detect environmental images. Specifically, it is necessary to collect a sufficient number of sample environmental images and label these sample environmental images to obtain sample environmental images with labels of the external environment of the vehicle window. Among them, labeling the sample environmental images mainly involves labeling the target objects in the sample environmental images. In the embodiments of the present application, it is mainly to label the sundries, soot, or smoke, etc. in the sample environmental images. Further, use these sample environmental images with labels of the external environment of the vehicle window to train the environmental detection model;

[0080] Further, detecting the environmental information in the environmental image through the environmental detection model can specifically be detecting whether the environmental image contains target objects through the environmental detection model. In the present application, the target objects can be at least one of sundries, soot, or smoke.

[0081] Further, if a target object is detected in the environmental image, then execute the window control strategy corresponding to detecting a target object in the environmental image; if no target object is detected in the environmental image, then execute the window control strategy corresponding to detecting no target object in the environmental image. For example: if at least one of sundries, soot, or smoke is detected in the environmental image, then execute the window control strategy corresponding to at least one of sundries, soot, or smoke in the environmental image; if none of sundries, soot, or smoke is detected in the environmental image, then execute the window control strategy corresponding to none of sundries, soot, or smoke in the environmental image. It should be noted that the window described in the present application can be the side window located at the vehicle door or the sunroof located at the vehicle top, and the present application does not make a limitation.

[0082] In a preferred embodiment, the inputting the environmental image into the environmental detection model to obtain the environmental information outside the vehicle window includes:

[0083] Input the environmental image into the environmental detection model to identify whether at least one target object in the environmental image is at least one of sundries, soot, or smoke;

[0084] If at least one of the target objects is at least one of the sundries, the soot, or the smoke, determine that the environmental information is abnormal environmental information;

[0085] If at least one of the target objects is not any of the sundries, the soot, or the smoke, determine that the environmental information is normal environmental information.

[0086] Specifically, in this embodiment, the environmental information is classified into abnormal environmental information and normal environmental information. Specifically, it is detected by the environmental detection model whether at least one target object in the environmental image collected by the camera is at least one of sundries, soot, or smoke; if it is detected that at least one target object in the environmental image is at least one of sundries, soot, or smoke, the environmental information in the environmental image is determined as abnormal environmental information; if it is detected that at least one target object in the environmental image is not any of sundries, soot, or smoke, the environmental information in the environmental image is determined as normal environmental information. It should be noted that the target objects being sundries, soot, or smoke are only partial examples proposed in this application. The target objects can also be other things such as vehicle exhaust, sand and dust, etc., which are not limited in this application.

[0087] Further, the executing a window control strategy corresponding to the environmental information in response to the environmental information includes:

[0088] When the environmental information is the abnormal environmental information and the window is in the open state, control the window to close;

[0089] When the environmental information is the normal environmental information and the window is in the closed state, control the window to open.

[0090] Specifically, in this embodiment, if the environmental detection model detects that the environmental information in the environmental image is abnormal environmental information, it indicates that there may be sundries, soot or smoke in the environment around the current vehicle, and the detection result is output to the vehicle-mounted system. The vehicle-mounted system responds to the detection result of the environmental detection model and executes the control strategy of closing the window. Specifically, the vehicle-mounted system first determines the opening and closing state of the window. If the window is in the open state, the vehicle-mounted system controls the window to close; if the window is already in the closed state, the vehicle-mounted system controls to keep the window in the closed state, so as to effectively prevent sundries, soot or smoke from entering the vehicle, bringing a bad driving experience to the driver or passengers, and at the same time effectively preventing the driver from being distracted during driving. If the environmental detection model detects that the environmental information in the environmental image is normal environmental information, it indicates that there are no sundries, soot or smoke in the environment around the current vehicle, and the detection result is output to the vehicle-mounted system. The vehicle-mounted system responds to the detection result of the environmental detection model and executes the control instruction of opening the window. Specifically, the vehicle-mounted system first determines the opening and closing state of the window. If the window is in the closed state, the vehicle-mounted system controls the window to open; if the window is already in the open state, the vehicle-mounted system controls to keep the window in the open state.

[0091] In another preferred embodiment, the step of inputting the environmental image into the environmental detection model to obtain the environmental information outside the vehicle window includes:

[0092] Inputting the environmental image into the environmental detection model to identify the weather information in the environmental image.

[0093] Specifically, in this application, not only can the environmental detection model be used to detect the target objects in the environmental image to obtain the environmental information in the environmental image, but also the environmental detection model can be used to detect the weather in the environmental image to obtain the weather information in the environmental image. Specifically, an environmental detection model capable of identifying the weather can be pre-trained: for example, an environmental detection model capable of identifying weather such as sunny, rainy, snowy days, etc., and the environmental image of the vehicle surroundings collected by the camera is input into the environmental detection model to detect the weather information in the environmental image.

[0094] It should be noted that in this application, an environmental detection model can be trained to simultaneously detect the target objects and weather information in the environmental image. Specifically, the labels of the target objects and weather information can be added to the training dataset, and then these labels are used when training the model. During training, multi-task learning or joint training methods can be used to train the model simultaneously, so that the trained environmental detection model can detect the target objects and weather information at the same time.

[0095] As Figure 2 shown is a flowchart of a window control method provided by an embodiment of this application. AsFigure 2 As shown, after the vehicle is powered on, the driver sets the in-vehicle window control function to be enabled. After the function is enabled, the camera captures the environmental images around the vehicle. The environmental detection model detects whether the environmental information in the environmental images captured by the camera is abnormal environmental information or whether it is abnormal weather. If the environmental detection model detects that the environmental information in the environmental image is abnormal environmental information and / or the environmental information is abnormal weather, the window is controlled to close. If the environmental detection model detects that the environmental information is neither abnormal environmental information nor abnormal weather, the window is controlled to open.

[0096] Further, the executing the window control strategy corresponding to the environmental information in response to the environmental information includes:

[0097] When the weather information represents abnormal weather and the window is in the open state, controlling the window to close;

[0098] When the weather information represents normal weather and the window is in the closed state, controlling the window to open.

[0099] Specifically, in this embodiment, if the environmental detection model detects that the weather information in the environmental image represents abnormal weather and outputs the detection result to the vehicle-mounted system, the vehicle-mounted system responds to the detection result of the environmental detection model and executes the control strategy of closing the window. Specifically, the vehicle-mounted system first judges the opening and closing state of the window. If the window is in the open state, the vehicle-mounted system controls the window to close; if the window is already in the closed state, the vehicle-mounted system controls to keep the window in the closed state. Among them, the environmental detection model detecting that the weather information in the environmental image represents abnormal weather specifically means that the environmental detection model detects that the weather information in the environmental image is bad weather such as rainy days and snowy days; if the environmental detection model detects that the weather represented in the environmental image is normal weather and outputs the detection result to the vehicle-mounted system, the vehicle-mounted system responds to the detection result of the environmental detection model and executes the control instruction of opening the window. Specifically, the vehicle-mounted system first judges the opening and closing state of the window. If the window is in the closed state, the vehicle-mounted system controls the window to open; if the window is already in the open state, the vehicle-mounted system controls to keep the window in the open state. Among them, the environmental detection model detecting that the weather represented in the environmental image is normal weather specifically means that the environmental detection model detects that the weather in the environmental image is sunny days and other weather.

[0100] In another preferred embodiment, it is determined whether a control instruction of the driver for the window is detected;

[0101] When the control instruction of the driver for the window is detected, stop executing the window control strategy corresponding to the environmental information;

[0102] In response to the driver's control instruction, execute a window control strategy corresponding to the driver's control instruction.

[0103] In this embodiment, during the process of executing the window control strategy corresponding to the environmental information, for example: when the environmental information is abnormal environmental information and the window is in the open state, during the process of controlling the window to close, or when the weather information represents abnormal weather and the window is in the open state, during the process of controlling the window to close, when detecting the driver's control instruction for the window, the vehicle-mounted system will stop executing the window control strategy corresponding to the environmental information. The driver's control instruction can be a control instruction triggered by the driver manually raising or pressing the window control button, and then respond to the control instruction triggered by the driver manually raising or pressing the window control button, and execute the control instruction manually triggered by the driver.

[0104] Or, when the environmental information is normal environmental information and the window is in the closed state, during the process of controlling the window to open, or when the weather information represents normal weather and the window is in the closed state, during the process of controlling the window to open, when detecting the driver's control instruction for the window, the vehicle-mounted system will stop executing the window control strategy corresponding to the environmental information. The driver's control instruction can be a control instruction triggered by the driver manually raising or pressing the window control button, and then respond to the control instruction triggered by the driver manually raising or pressing the window control button, and execute the control instruction manually triggered by the driver.

[0105] In this embodiment, during the process of executing the window control strategy corresponding to the environmental information, when detecting the driver's control instruction for the window, give priority to executing the driver's control instruction for the window, which can avoid executing the wrong window control instruction due to the detection error of the environmental detection model.

[0106] In yet another preferred embodiment, the responding to the environmental information and executing a window control strategy corresponding to the environmental information includes:

[0107] When the weather information represents abnormal weather, control the vehicle's voice system to emit a first voice, and the first voice is used to prompt the driver that the weather is abnormal and ask whether to choose to close the window;

[0108] When the weather information represents normal weather, control the vehicle's voice system to emit a second voice, and the second voice is used to prompt the driver whether to choose to open the window or whether to choose to turn on the air conditioner;

[0109] In response to the detected feedback instruction of the driver for the first voice or the second voice, execute a window control strategy corresponding to the feedback instruction.

[0110] In this embodiment, the vehicle's voice system can be incorporated into the window control strategy corresponding to the environmental information. Specifically, if the environmental detection model detects that the weather information in the environmental image represents abnormal weather, the environmental detection model first outputs the detection result to the vehicle-mounted system. The vehicle-mounted system controls the vehicle's voice system to emit a first voice corresponding to the detection result according to the detection result output by the environmental detection model. The first voice is used to prompt the driver that the weather is abnormal and ask whether to choose to close the window. After receiving the first voice emitted by the voice system, the driver issues a feedback instruction corresponding to the first voice. The feedback instruction can be that the driver emits a voice to tell the vehicle-mounted system their intention. For example, the driver can say, "Close the window." After receiving the driver's feedback instruction, the vehicle's voice system converts it into text that the vehicle-mounted system can recognize and sends the recognized text to the vehicle-mounted system. The vehicle-mounted system controls the window according to the driver's instruction in the recognized text. For example, if the driver's instruction in the recognized text is the instruction to "close the window", the vehicle-mounted system will control the window to close according to the instruction to close the window.

[0111] Exemplarily, when the environmental detection model detects that the weather information in the environmental image is rainy, the vehicle-mounted system controls the voice system to emit the first voice: "It is raining currently. Do you choose to close the window?" The driver answers with the voice "Close the window". After the voice system converts the collected voice of the driver into text that the vehicle-mounted system can recognize, the vehicle-mounted system controls the window to close according to the instruction in the text.

[0112] If the environmental detection model detects that the weather information in the environmental image represents normal weather, the environmental detection model first outputs the detection result to the vehicle-mounted system. The vehicle-mounted system emits a second voice corresponding to the detection result according to the detection result output by the environmental detection model. The second voice is used to prompt the driver whether to choose to open the window or turn on the air conditioner. After receiving the second voice emitted by the voice system, the driver issues a feedback instruction corresponding to the second voice. The feedback instruction can be that the driver emits a voice to tell the vehicle-mounted system their intention. For example: The driver can say, "Open the window" or "Turn on the air conditioner". After receiving the driver's feedback instruction, the vehicle's voice system converts it into text that the vehicle-mounted system can recognize and sends the recognized text to the vehicle-mounted system. The vehicle-mounted system controls the window according to the driver's instruction in the recognized text. For example: If the driver's instruction in the recognized text is the instruction to "open the window", the vehicle-mounted system will control the window to open, or, if the driver's instruction in the recognized text is the instruction to "turn on the air conditioner", the vehicle-mounted system will control the air conditioner to turn on.

[0113] Exemplarily, when the environmental detection model detects that the weather information in the environmental image is sunny, the in-vehicle system controls the voice system to issue a second voice: "The current weather is clear. Do you want to open the window or turn on the air conditioner?"; if the driver's voice response is "Open the window", after the voice system converts the driver's voice collected into text that the in-vehicle system can recognize, the in-vehicle system controls the window to open according to the instruction in the text. If the driver's voice response is "Turn on the air conditioner", after the voice system converts the driver's voice collected into text that the in-vehicle system can recognize, the in-vehicle system controls the air conditioner to turn on according to the instruction in the text.

[0114] In this embodiment, by adding the vehicle's voice system to the window control strategy corresponding to the environmental information, the interactivity of the in-vehicle system is greatly improved, and the driving experience of the driver is enhanced.

[0115] In another preferred embodiment, the environmental detection model is obtained by inputting the multiple sample environmental images with the environmental labels outside the vehicle window into the YOLOv4 network for training.

[0116] Specifically, referring to Figure 3 , which is a schematic diagram of the YOLOv4 network structure of an environmental detection model framework provided by an embodiment of the present application. In this embodiment, to train the environmental detection model, a sufficient number of sample environmental images need to be collected and these sample environmental images need to be marked to obtain sample environmental images with environmental labels outside the vehicle window. Among them, marking the sample environmental images mainly involves marking the target objects in the sample environmental images. In the embodiments of the present application, it is mainly to mark the sundries, soot or smoke, etc. in the sample environmental images. Further, these sample environmental images with environmental labels outside the vehicle window are used to train the environmental detection model. In the present application, the YOLOv4 network is selected as the basic model network to train the environmental detection model.

[0117] Preferably, the environmental detection model uses Mobilenetv2 as the backbone network to reduce the number of parameters and the computational amount of the environmental detection model, and / or, the Resunit (Residual Unit) block of the environmental detection model is a module with an ECA (Efficient Channel Attention) attention mechanism, which is used to improve the attention of the environmental detection model to the target objects in the environmental image.

[0118] Specifically, in this embodiment, the BackBone in the original YOLOv4 network is replaced with Mobilenetv2 with fewer parameters, so as to reduce the number of parameters of the environmental detection model and the computational amount during the detection process, and / or an ECA attention mechanism is added to all Resunit blocks in the original YOLOv4 network to improve the attention of the environmental detection model to the target objects in the environmental image.

[0119] Specifically, referring to Figure 4 the schematic diagram of the ECA attention mechanism shown, the environmental image is recognized by the YOLOv4 network with the added ECA attention mechanism. First, a series of feature maps with different sizes and different numbers of channels of the environmental image are obtained according to the processing of the BackBone part of the YOLOv4 network. Taking one of the feature maps as an example, this feature map can represent a certain feature in the environmental image, such as one of sundries, soot or smoke, or it can be sunny, rainy, snowy, etc. This feature map is a three-dimensional tensor of H*W*C, where H represents the height, W represents the width, and C represents the number of channels. Spatial feature compression is performed on the said feature map, specifically, it can be operated by average pooling to obtain a 1*1*C feature map, that is, the spatial information is compressed into channel information.

[0120] Furthermore, channel feature learning is carried out. Specifically, channel feature learning is performed on the compressed feature map. The importance between different channels is learned through a 1*1 convolutional layer, and the learned channel attention coefficients are applied to each channel of the compressed feature map to obtain a compressed feature map with channel attention. It should be noted that when performing the 1*1 convolution operation, we use a dynamic convolutional kernel to adapt to the receptive field size of different input feature maps to extract features in different ranges. Specifically, in the layer with a larger number of channels, we use a larger convolutional kernel to achieve cross-channel interaction, while in the layer with a smaller number of channels, we use a smaller convolutional kernel to reduce cross-channel interaction. Among them, the size of the dynamic convolutional kernel is determined by a convolutional kernel adaptive function, and the said adaptive function is shown in the following formula:

[0121]

[0122] where k is the convolutional kernel size; C is the number of channels; |·| odd means that k can only take odd numbers; γ and b are constants, usually γ is set to 2 and b is set to 1.

[0123] Furthermore, the compressed feature map with channel attention is multiplied by the original feature map channel by channel, and finally a feature map with channel attention is output.

[0124] In this embodiment, a 1*1 convolutional layer is used to capture information between different channels, avoiding the reduction of the channel dimension when learning channel attention information, reducing the number of parameters. Adding the ECA attention mechanism can make the network pay more attention to the target to be detected without additionally increasing the number of model parameters, improving the detection effect, and thus achieving a performance improvement.

[0125] The Resunit block is a commonly used neural network structure in deep learning. The data processing process of a neural network model constructed using the Resunit block generally involves processing the data input into the model through a convolutional layer, usually including: convolutional operation, batch normalization operation, and activation function processing, etc. Then, the processed data is further processed through a second convolutional layer, which also includes: convolutional operation, batch normalization operation, and activation function processing, etc. The data obtained by adding the output of the second convolutional layer and the input data is used as the final output data of the Resunit block.

[0126] As Figure 5 shown in the schematic diagram of the network structure with the ECA attention mechanism added to the Resunit block, the ECA attention mechanism is added after two CBM (Conv Batchnorm Mish convolutional batch normalization activation function) blocks respectively. After adding the ECA attention mechanism, an additional step will be added to the data processing process of the Resunit block, that is, applying the ECA attention mechanism to the output data of the Resunit block. The main purpose of adding the ECA attention mechanism to the Resunit block is to improve the network's learning ability for different channel features and better capture the interaction between features. The ECA attention mechanism can adaptively learn the importance of each channel, thereby performing weighted fusion on the features of each channel in the residual structure of the Resunit block, improving the network's expressive ability, and further improving the recognition performance of the environment detection model. When the environment detection model recognizes the environmental image around the vehicle, it improves the attention to the target object, and at the same time improves the generalization ability and robustness of the environment detection model.

[0127] An embodiment of the present application provides a window control method, and the method includes: obtaining an environmental image around the vehicle; inputting the environmental image into an environmental detection model to obtain environmental information outside the vehicle, where the environmental detection model is trained based on a plurality of sample environmental images with environmental labels outside the vehicle; and in response to the environmental information, executing a window control strategy corresponding to the environmental information. The window control method provided by the present application determines the environmental information outside the vehicle by detecting the environmental image around the vehicle and executes a window control strategy corresponding to the environmental information, without the need for the driver to manually control the window button to control the opening and closing of the window. On the one hand, it avoids the problem of the driver being distracted while driving and causing traffic accidents; on the other hand, it can sense the environmental information around the vehicle and control the window according to the environmental information. It improves the driving safety and experience of the driver.

[0128] In the second aspect of the embodiment of the present application, a window control system is provided, as Figure 6 the schematic diagram of the window control system shown, and the system includes:

[0129] An acquisition module 201, configured to acquire an environmental image around the vehicle;

[0130] An input module 202, configured to input the environmental image into an environmental detection model to obtain environmental information outside the vehicle, where the environmental detection model is trained based on a plurality of sample environmental images with environmental labels outside the vehicle;

[0131] An execution module 203, configured to execute a window control strategy corresponding to the environmental information in response to the environmental information.

[0132] Optionally, for inputting the environmental image into the environmental detection model to obtain environmental information outside the vehicle, the input module 202 includes:

[0133] A first input sub-module, configured to input the environmental image into the environmental detection model to identify whether at least one target object in the environmental image is at least one of sundries, soot, or smoke;

[0134] A first determination sub-module, configured to determine that the environmental information is abnormal environmental information if the at least one target object is at least one of the sundries, the soot, or the smoke;

[0135] A second determination sub-module, configured to determine that the environmental information is normal environmental information if the at least one target object is not any of the sundries, the soot, or the smoke.

[0136] Optionally, for inputting the environmental image into the environmental detection model to obtain environmental information outside the vehicle, the input module 202 includes:

[0137] A second input sub-module, configured to input the environmental image into the environmental detection model to identify weather information in the environmental image.

[0138] Optionally, in response to the environmental information, implementing a window control strategy corresponding to the environmental information, the execution module 203 includes:

[0139] A first control sub-module, configured to control the vehicle's voice system to emit a first voice when the weather information represents abnormal weather, where the first voice is used to prompt the driver that the weather is abnormal and ask whether to close the window;

[0140] A second control sub-module, configured to control the vehicle's voice system to emit a second voice when the weather information represents normal weather, where the second voice is used to prompt the driver whether to open the window or whether to turn on the air conditioner;

[0141] A first execution sub-module, configured to, in response to a feedback instruction detected from the driver for the first voice or the second voice, implement a window control strategy corresponding to the feedback instruction.

[0142] Optionally, in response to the environmental information, implementing a window control strategy corresponding to the environmental information, the execution module 203 includes:

[0143] A third control sub-module, configured to control the window to close when the environmental information is the abnormal environmental information and the window is in an open state;

[0144] A fourth control sub-module, configured to control the window to open when the environmental information is the normal environmental information and the window is in a closed state.

[0145] Optionally, in response to the environmental information, implementing a window control strategy corresponding to the environmental information, the execution module 203 includes:

[0146] A fifth control sub-module, configured to control the window to close when the weather information represents abnormal weather and the window is in an open state;

[0147] A sixth control sub-module, configured to control the window to open when the weather information represents normal weather and the window is in a closed state.

[0148] Optionally, during the process of implementing a window control strategy corresponding to the environmental information, it further includes:

[0149] A third determination sub-module, configured to determine whether a control instruction of the driver for the window is detected;

[0150] A monitoring sub-module, configured to stop executing the window control strategy corresponding to the environmental information when detecting the driver's control instruction for the window;

[0151] A second execution sub-module, configured to execute the window control strategy corresponding to the driver's control instruction in response to the driver's control instruction.

[0152] Optionally, the environmental detection model is obtained by training the YOLOv4 network by inputting the multiple sample environmental images with the environmental labels outside the vehicle window.

[0153] Optionally, the environmental detection model uses Mobilenetv2 as the backbone network to reduce the number of parameters and the computational amount of the environmental detection model, and / or,

[0154] The Resunit block of the environmental detection model is a module with an ECA attention mechanism, configured to improve the attention of the environmental detection model to the target object in the environmental image.

[0155] In a third aspect of the embodiments of the present application, a vehicle is provided, and the vehicle includes the window control system as described in the second aspect of the embodiments of the present application.

[0156] In each embodiment of this specification, the key points described are the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other.

[0157] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a device, or a computer program product. Therefore, the embodiments of the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0158] Embodiments of the present application are described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in one or more flows Figure 1 one or more flows and / or blocks Figure 1 or a plurality of blocks.

[0159] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in one or more flows Figure 1 one or more flows and / or blocks Figure 1 or a plurality of blocks.

[0160] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more flows Figure 1 one or more flows and / or blocks Figure 1 or a plurality of blocks.

[0161] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they know the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0162] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising said element.

[0163] The above provides a detailed introduction to a window control method, system and vehicle. Specific examples are used in this text to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A window control method, characterized in that, The method includes: Obtaining an environmental image around the vehicle; Inputting the environmental image into an environmental detection model to obtain environmental information outside the vehicle window, where the environmental detection model is trained based on multiple sample environmental images with environmental labels outside the vehicle window; Responding to the environmental information and executing a window control strategy corresponding to the environmental information; Among them, the step of inputting the environmental image into the environmental detection model to obtain environmental information outside the vehicle window includes: Inputting the environmental image into the environmental detection model to identify weather information in the environmental image; The step of responding to the environmental information and executing a window control strategy corresponding to the environmental information includes: When the weather information represents abnormal weather, controlling the vehicle's voice system to emit a first voice, where the first voice is used to prompt the driver that the weather is abnormal and ask whether to close the window; When the weather information represents normal weather, controlling the vehicle's voice system to emit a second voice, where the second voice is used to prompt the driver whether to open the window or whether to turn on the air conditioner; Responding to the detected feedback instruction of the driver for the first voice or the second voice and executing a window control strategy corresponding to the feedback instruction.

2. The window control method according to claim 1, wherein The step of inputting the environmental image into the environmental detection model to obtain environmental information outside the vehicle window includes: Inputting the environmental image into the environmental detection model to identify whether at least one target object in the environmental image is at least one of sundries, soot, or smoke; If the at least one target object is at least one of the sundries, the soot, or the smoke, determining that the environmental information is abnormal environmental information; If the at least one target object is not any of the sundries, the soot, or the smoke, determining that the environmental information is normal environmental information.

3. The window control method according to claim 2, characterized in that, The step of responding to the environmental information and executing a window control strategy corresponding to the environmental information includes: When the environmental information is the abnormal environmental information and the window is in the open state, controlling the window to close; When the environmental information is the normal environmental information and the window is in the closed state, controlling the window to open.

4. The window control method according to claim 1, wherein The step of responding to the environmental information and executing a window control strategy corresponding to the environmental information includes: When the weather information represents abnormal weather and the window is in the open state, controlling the window to close; When the weather information represents normal weather and the window is in the closed state, controlling the window to open.

5. The window control method according to claim 1, characterized in that During the process of executing the window control strategy corresponding to the environmental information, it further includes: Determining whether a control instruction of the driver for the window is detected; When a control instruction of the driver for the window is detected, stopping the execution of the window control strategy corresponding to the environmental information; Responding to the control instruction of the driver and executing a window control strategy corresponding to the control instruction of the driver.

6. The window control method according to any one of claims 1-5, characterized in that, The environmental detection model is obtained by inputting the multiple sample environmental images with the environmental labels outside the vehicle window into the YOLOv4 network for training.

7. The window control method according to claim 6, wherein The environmental detection model uses Mobilenetv2 as the backbone network to reduce the number of parameters and the computational amount of the environmental detection model, and / or the Resunit block of the environmental detection model is a module with an ECA attention mechanism, which is used to improve the attention of the environmental detection model to the target objects in the environmental image.

8. A window control system, characterized in that, The system includes: an acquisition module, configured to acquire environmental images around the vehicle; an input module, configured to input the environmental images into an environmental detection model to obtain environmental information outside the vehicle window, where the environmental detection model is obtained by training based on multiple sample environmental images with environmental labels outside the vehicle window; an execution module, configured to execute a window control strategy corresponding to the environmental information in response to the environmental information; The input module includes: a second input sub-module, configured to input the environmental images into the environmental detection model to identify weather information in the environmental images; The execution module includes: a first control sub-module, configured to control the voice system of the vehicle to emit a first voice when the weather information represents abnormal weather, where the first voice is used to prompt the driver that the weather is abnormal and whether to choose to close the window; a second control sub-module, configured to control the voice system of the vehicle to emit a second voice when the weather information represents normal weather, where the second voice is used to prompt the driver whether to choose to open the window or whether to choose to turn on the air conditioner; a first execution sub-module, configured to execute a window control strategy corresponding to the feedback instruction in response to the detected feedback instruction of the driver for the first voice or the second voice.

9. A vehicle, characterized in that, The vehicle includes the window control system according to claim 8.

Images