In-vehicle environment monitoring method, in-vehicle environment monitoring device, equipment and medium
By installing cameras in the car and monitoring the environment in the car using detection models, the safety hazards of children or pets doing dangerous behaviors in the car are solved, and the life safety of children and pets is guaranteed.
Patent Information
- Application Number
- CN202411938564.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-04-18
AI Technical Summary
During the car driving, children or pets may perform dangerous behaviors without being monitored in the car, which poses great safety hazards.
The in-vehicle environment images are collected by cameras installed in multiple preset positions in the car, and the object detection model is used to detect whether there are movable children or pets. When the vehicle is driving and the hazardous action monitoring function is turned on, the hazardous action detection model is used to monitor and generate alarm information in real time.
It effectively solves the potential dangers caused by moving children or pets in the car in the seat, and protects the lives of children and pets.
Smart Images
Figure CN119851250A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of intelligent automobile technology, and in particular to a method, a device, equipment and a medium for monitoring an in-vehicle environment. Background Art
[0002] With the improvement of living standards, the use of cars is becoming more and more popular, and gradually entering more and more families. In daily life, people will drive out with children or even pets.
[0003] However, if children or pets are allowed to move around in the car and are not supervised while driving, they are likely to engage in dangerous behaviors, such as jumping in the back seat, climbing on seats, opening car doors, or sticking their heads out of the window, posing a major safety hazard. Summary of the invention
[0004] In view of this, the purpose of this application is to provide a method, device, equipment and medium for monitoring the in-vehicle environment, which uses a camera to collect in-vehicle environment images, and uses an object detection model to detect whether there are unfixed and freely moving children or pets in the current vehicle. When the dangerous action monitoring function is turned on, the dangerous action detection model is used to monitor in real time whether the children or pets have dangerous actions, and if so, a corresponding alarm is issued. The potential danger caused by the movement of children or pets in the seats in the vehicle is effectively solved, and the life safety of children and pets is guaranteed.
[0005] In a first aspect, an embodiment of the present application provides a method for monitoring an in-vehicle environment, the method comprising:
[0006] For each camera installed at a plurality of preset positions in the vehicle, an image of the vehicle interior environment captured by the camera is obtained;
[0007] Inputting the in-vehicle environment image into an object detection model, and generating prompt information when the object detection model detects that there is a movable target object in the in-vehicle environment image; wherein the movable target object is a child or a pet;
[0008] When a dangerous action monitoring instruction is received and it is detected that the vehicle is moving, the interior environment image captured by the camera is input into the dangerous action detection model. When the dangerous action detection model detects that a movable target object in the interior environment image has dangerous behavior, a first alarm message is generated.
[0009] Furthermore, when the dangerous action detection model detects that the movable target object in the in-vehicle environment image has dangerous behavior, the in-vehicle environment monitoring method further includes:
[0010] The behavior type corresponding to the dangerous behavior is determined based on the dangerous action detection model, and the second alarm information is generated based on the behavior type.
[0011] Furthermore, the multiple preset positions include a co-pilot position, a left side position of the rear seat, and a right side position of the rear seat.
[0012] Furthermore, when a dangerous action monitoring instruction is received and it is detected that the vehicle is driving, the in-vehicle environment monitoring method further includes:
[0013] Determine a target camera that captures an image of the vehicle interior environment where a movable target object exists;
[0014] Determine the target window and the target door corresponding to the target camera, and generate a forced locking instruction for the target window and the target door;
[0015] The forced locking instruction is sent to a vehicle body control module, so that the vehicle body control module forces the target vehicle window and the target vehicle door to be locked.
[0016] Furthermore, the dangerous action detection model is trained by the following steps:
[0017] Acquire multiple dangerous action sample images; wherein each dangerous action sample image has a corresponding dangerous action sample type;
[0018] Inputting a plurality of the dangerous action sample images into a dangerous action original detection model to determine a dangerous action prediction type corresponding to each dangerous action sample image;
[0019] Compare the dangerous action sample type corresponding to each dangerous action sample image with the dangerous action prediction type, and calculate the loss function of the original detection model of the dangerous action in the current state;
[0020] Based on the loss function of the original dangerous action detection model, the model parameters of the original dangerous action detection model are continuously adjusted until the original dangerous action detection model reaches a convergence state, thereby obtaining the dangerous action detection model.
[0021] Furthermore, the object detection model is trained by the following steps:
[0022] Acquire a plurality of object sample images; wherein the plurality of object sample images include a plurality of sample images containing movable objects and a plurality of sample images containing immovable objects, and each object sample image corresponds to a label value;
[0023] Inputting a plurality of the object sample images into an original object detection model to determine a prediction value corresponding to each object sample image;
[0024] Compare the label value corresponding to each object sample image with the predicted value, and calculate the loss function of the original detection model of the object in the current state;
[0025] Based on the loss function of the original object detection model, the model parameters of the original object detection model are continuously adjusted until the original object detection model reaches a convergence state, thereby obtaining the object detection model.
[0026] In a second aspect, an embodiment of the present application further provides an in-vehicle environment monitoring device, the in-vehicle environment monitoring device comprising:
[0027] An image acquisition module, for acquiring an image of the vehicle interior environment captured by each camera installed at a plurality of preset positions in the vehicle;
[0028] A first detection module is used to input the in-vehicle environment image into an object detection model, and generate prompt information when the object detection model detects that there is a movable target object in the in-vehicle environment image; wherein the movable target object is a child or a pet;
[0029] The second detection module is used to input the interior environment image captured by the camera into the dangerous action detection model when a dangerous action monitoring instruction is received and it is detected that the vehicle is moving, and generate a first alarm message when the dangerous action detection model detects that a movable target object in the interior environment image has dangerous behavior.
[0030] Furthermore, the in-vehicle environment monitoring device further includes an information generation module. When the dangerous action detection model detects that the movable target object in the in-vehicle environment image has dangerous behavior, the information generation module is used to:
[0031] The behavior type corresponding to the dangerous behavior is determined based on the dangerous action detection model, and the second alarm information is generated based on the behavior type.
[0032] In a third aspect, an embodiment of the present application also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the in-vehicle environment monitoring method as described above are performed.
[0033] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the in-vehicle environment monitoring method as described above are executed.
[0034] The embodiments of the present application provide a method, device, equipment and medium for monitoring an in-vehicle environment. First, for each camera installed at a plurality of preset positions in a vehicle, an in-vehicle environment image captured by the camera is acquired; then, the in-vehicle environment image is input into an object detection model, and when the object detection model detects that there is a movable target object in the in-vehicle environment image, a prompt message is generated; wherein the movable target object is a child or a pet; finally, when a dangerous action monitoring instruction is received and it is detected that the vehicle is driving, the in-vehicle environment image captured by the camera is input into the dangerous action detection model, and when the dangerous action detection model detects that there is a dangerous behavior of the movable target object in the in-vehicle environment image, a first alarm message is generated.
[0035] This application uses a camera to collect images of the vehicle's interior environment, and uses an object detection model to detect whether there are any unsecured, freely moving children or pets in the vehicle. When the dangerous action monitoring function is turned on, the dangerous action detection model is used to monitor in real time whether children or pets are making dangerous movements, and if so, a corresponding alarm is issued. This effectively solves the potential dangers caused by children or pets moving in their seats in the vehicle, and ensures the safety of children and pets.
[0036] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0038] Figure 1 A flow chart of a method for monitoring an in-vehicle environment provided in an embodiment of the present application;
[0039] Figure 2 One of the structural schematic diagrams of a vehicle interior environment monitoring device provided in an embodiment of the present application;
[0040] Figure 3 The second structural diagram of a vehicle interior environment monitoring device provided in an embodiment of the present application;
[0041] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0042] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.
[0043] First, the application scenarios to which the present application is applicable are introduced. The present application can be applied in the field of intelligent vehicle technology.
[0044] With the improvement of living standards, the use of cars is becoming more and more popular, and gradually entering more and more families. In daily life, people will drive out with children or even pets.
[0045] However, if children or pets are allowed to move around in the car and are not supervised while driving, they are likely to engage in dangerous behaviors, such as jumping in the back seat, climbing on seats, opening car doors, or sticking their heads out of the window, posing a major safety hazard.
[0046] Based on this, an embodiment of the present application provides a method for monitoring the in-vehicle environment, which effectively solves the potential dangers caused by children or pets moving on seats in the car, and ensures the life safety of children and pets.
[0047] See also Figure 1 , Figure 1 This is a flow chart of a method for monitoring the in-vehicle environment provided by an embodiment of the present application. Figure 1 As shown in , the in-vehicle environment monitoring method provided by the embodiment of the present application includes:
[0048] S101, for each camera installed at a plurality of preset positions in the vehicle, an image of the vehicle interior environment captured by the camera is obtained.
[0049] Cameras are installed at multiple preset positions in the vehicle to capture images of the vehicle interior environment to monitor the vehicle interior environment. For the above step S101, in the specific implementation, for each camera installed at multiple preset positions in the vehicle, the image of the vehicle interior environment captured by the camera is obtained. Here, the image captured by the camera can be obtained through a video line, which is not specifically limited in this application.
[0050] Specifically, according to the embodiment provided by the present application, the plurality of preset positions include the co-pilot position, the left position of the rear seat, and the right position of the rear seat. Here, the camera installed at the co-pilot position is used to capture the environment image of the co-pilot, the camera installed at the left position of the rear seat is used to capture the environment image of the left side of the rear seat, and the camera installed at the right position of the rear seat is used to capture the environment image of the right side of the rear seat.
[0051] S102, inputting the in-vehicle environment image into an object detection model, and generating prompt information when the object detection model detects that there is a movable target object in the in-vehicle environment image.
[0052] Here, the movable target object refers to a target object that is not fixed and can move freely. Specifically, the movable target object is a child or a pet. For example, for younger children, a safety seat is often placed in the car to protect the child's safety during the vehicle's driving. Therefore, a child who is not in a safety seat and can move freely is a movable target object. For pets, a pet cage is placed in the car to prevent the pet from moving freely in the car, thereby improving driving safety. Therefore, a pet that is not in a pet cage and can move freely is a movable target object.
[0053] Regarding the above step S102, in the specific implementation, for each camera, the in-vehicle environment image captured by the camera is input into a pre-trained object detection model. When the object detection model detects that there is a movable target object in the input in-vehicle environment image, that is, there are movable children or pets in the car, a prompt message is generated to remind the user. Specifically, the prompt message can be directly displayed on the in-vehicle display screen, or the prompt message can be played by voice, which is not specifically limited in this application.
[0054] As an optional embodiment, according to the in-vehicle environment monitoring method provided by the present application, the object detection model is trained through the following steps:
[0055] A: Get multiple object sample images.
[0056] B: Inputting the plurality of object sample images into the original object detection model to determine the prediction value corresponding to each object sample image.
[0057] Here, the multiple object sample images include multiple sample images with movable objects and multiple sample images with immovable objects, and each object sample image corresponds to a label value. Specifically, since the object detection model is used to detect whether there are movable children or pets in the image, the label value corresponding to the sample image with movable objects is "existent", and the label value corresponding to the sample image with immovable objects is "non-existent".
[0058] Regarding the above-mentioned steps A and B, during the specific implementation, a plurality of object sample images are obtained, and the plurality of object sample images are input into the original object detection model to determine the prediction value corresponding to each object sample image.
[0059] C: Compare the label value corresponding to each object sample image with the predicted value, and calculate the loss function of the original detection model of the object in the current state.
[0060] D: Based on the loss function of the original object detection model, the model parameters of the original object detection model are continuously adjusted until the original object detection model reaches a convergence state to obtain the object detection model.
[0061] It should be noted that the loss function is a function that maps the value of a random event or its related random variables to a non-negative real number to represent the "risk" or "loss" of the random event. In applications, the loss function is usually associated with the optimization problem as a learning criterion, that is, the model is solved and evaluated by minimizing the loss function.
[0062] For the above steps C-D, in the specific implementation, for each object sample image, the label value corresponding to the object sample image is compared with the predicted value, and the prediction of the object original detection model is judged by comparing the label value corresponding to the object sample image with the predicted value. When the label value corresponding to the object sample image is different from the predicted value, it is considered that the prediction of the object original detection model is inaccurate. At this time, it is necessary to calculate the loss function of the object original detection model in the current state. The method of calculating the loss function is described in detail in the prior art, and will not be described in detail here. Then, based on the calculated loss function, the model parameters of the object original detection model are continuously adjusted. The object original detection model will continuously minimize the loss in an iterative manner. At each step of the iteration, the loss value of the object original detection model will be calculated. When the loss value of the object original detection model cannot reach the loss threshold, the model parameters of the object original detection model are continuously updated, and the new parameters will calculate a new loss value, so that the loss value shows a fluctuating downward trend during the iteration process. Finally, when the loss value reaches smoothness, that is, the loss value of the trained object original detection model is not greater than the loss threshold, that is, when the loss value does not decrease significantly compared with the loss value calculated last time, the object original detection model is considered to have reached a convergence state. At this time, the prediction of the object original detection model is relatively accurate. At this time, the training is ended and the object detection model is obtained.
[0063] S103, when a dangerous action monitoring instruction is received and it is detected that the vehicle is moving, the in-vehicle environment image captured by the camera is input into a dangerous action detection model, and when the dangerous action detection model detects that a movable target object in the in-vehicle environment image has dangerous behavior, a first alarm message is generated.
[0064] Here, dangerous actions include children or pets opening car windows, doors, moving inside the car, leaning out of the window, extending the body toward the driving operation area, etc. This application does not make specific limitations on this.
[0065] For the above step S103, in the specific implementation, when a movable target object is detected in the vehicle, the user can choose whether to turn on the dangerous action monitoring function. The user can choose by clicking the button on the vehicle display screen, or by voice. The specific method is not specifically limited in this application. When the dangerous action monitoring function is turned on, a dangerous action monitoring instruction will be received, and when it is detected that the vehicle is driving, the image of the vehicle interior environment captured by the camera is input into the dangerous action detection model. When the dangerous action detection model detects that the movable target object in the vehicle interior environment image has dangerous behavior, a first alarm message is generated to remind the user. Specifically, the first alarm message can be directly displayed on the vehicle display screen, or a prompt message can be played through voice. This application does not make specific limitations on this.
[0066] As an optional embodiment, when a dangerous action monitoring instruction is received and it is detected that the vehicle is driving, the in-vehicle environment monitoring method further includes:
[0067] (1) Determine the target camera that captures the image of the vehicle interior environment where the movable target object exists.
[0068] (2) Determine the target window and the target door corresponding to the target camera, and generate a forced locking instruction for the target window and the target door.
[0069] With respect to the above steps (1) to (2), in the specific implementation, when a dangerous action monitoring instruction is received and it is detected that the vehicle is moving, the target camera that captures the image of the interior environment of the vehicle with a movable target object is determined. Then the target windows and target doors corresponding to the target camera are determined, and a forced locking instruction for the target windows and target doors is generated. Here, as an example, when the target camera is installed in the co-pilot position, the target windows and target doors corresponding to the target camera are the windows and doors on the co-pilot side, and when the target camera is installed in the right side of the rear seat, the target windows and target doors corresponding to the target camera are the windows and doors on the right rear side of the vehicle.
[0070] (3) sending the forced locking instruction to a body control module, so that the body control module forces the target window and the target door to be locked.
[0071] With respect to the above step (3), in the specific implementation, after the forced locking instruction is generated, the forced locking instruction is sent to the body control module, so that the body control module forcibly locks the target windows and target doors based on the forced locking instruction. Here, the forced locking instruction can be sent to the body control module via the CAN bus, and this application does not make specific restrictions on this. In this way, when the dangerous action monitoring function is turned on and the vehicle is on the way, the windows and doors corresponding to the location of the children or pets can be forced to lock, protecting the safety of children and pets during driving.
[0072] As an optional embodiment, when the dangerous action detection model detects that the movable target object in the in-vehicle environment image has dangerous behavior, the in-vehicle environment monitoring method further includes:
[0073] The behavior type corresponding to the dangerous behavior is determined based on the dangerous action detection model, and the second alarm information is generated based on the behavior type.
[0074] In the above steps, when implemented, the dangerous action detection model can detect the type of dangerous actions in the image, determine the behavior type corresponding to the dangerous behavior of the child or pet based on the dangerous action detection model, and generate the second alarm information based on the behavior type. In this way, the user can learn about the dangerous behavior currently performed by the child or pet through the second alarm information and take timely measures to stop it.
[0075] As an optional embodiment, according to the in-vehicle environment monitoring method provided by the present application, the dangerous action detection model is trained through the following steps:
[0076] I: Obtain multiple dangerous action sample images.
[0077] Here, each dangerous action sample image has a corresponding dangerous action sample type.
[0078] II: Inputting a plurality of the dangerous action sample images into the original dangerous action detection model, and determining the dangerous action prediction type corresponding to each dangerous action sample image.
[0079] III: Compare the dangerous action sample type corresponding to each dangerous action sample image with the dangerous action prediction type, and calculate the loss function of the original detection model of the dangerous action in the current state.
[0080] IV: Based on the loss function of the original dangerous action detection model, the model parameters of the original dangerous action detection model are continuously adjusted until the original dangerous action detection model reaches a convergence state, thereby obtaining the dangerous action detection model.
[0081] Here, the process of training the dangerous action detection model, that is, the description of the above steps I to IV can refer to the description of steps A to D, and can achieve the same technical effect, which will not be repeated here.
[0082] The in-vehicle environment monitoring method provided in the embodiment of the present application first obtains the in-vehicle environment image captured by each camera installed at multiple preset positions in the vehicle; then, the in-vehicle environment image is input into the object detection model, and when the object detection model detects the presence of a movable target object in the in-vehicle environment image, a prompt message is generated; wherein the movable target object is a child or a pet; finally, when a dangerous action monitoring instruction is received and it is detected that the vehicle is driving, the in-vehicle environment image taken by the camera is input into the dangerous action detection model, and when the dangerous action detection model detects that the movable target object in the in-vehicle environment image has dangerous behavior, a first alarm message is generated.
[0083] This application uses a camera to collect images of the vehicle's interior environment, and uses an object detection model to detect whether there are any unsecured, freely moving children or pets in the vehicle. When the dangerous action monitoring function is turned on, the dangerous action detection model is used to monitor in real time whether children or pets are making dangerous movements, and if so, a corresponding alarm is issued. This effectively solves the potential dangers caused by children or pets moving in their seats in the vehicle, and ensures the safety of children and pets.
[0084] See also Figure 2 , Figure 3 , Figure 2 This is one of the structural schematic diagrams of a vehicle interior environment monitoring device provided in an embodiment of the present application. Figure 3 This is a second structural diagram of a vehicle interior environment monitoring device provided in an embodiment of the present application. Figure 2 As shown in , the in-vehicle environment monitoring device 200 includes:
[0085] The image acquisition module 201 is used to acquire the vehicle interior environment image captured by each camera installed at a plurality of preset positions in the vehicle;
[0086] The first detection module 202 is used to input the in-vehicle environment image into an object detection model, and generate prompt information when the object detection model detects that there is a movable target object in the in-vehicle environment image; wherein the movable target object is a child or a pet;
[0087] The second detection module 203 is used to input the interior environment image captured by the camera into the dangerous action detection model when a dangerous action monitoring instruction is received and it is detected that the vehicle is moving, and generate a first alarm message when the dangerous action detection model detects that a movable target object in the interior environment image has dangerous behavior.
[0088] Further, such as Figure 3 As shown, the in-vehicle environment monitoring device 200 further includes an information generating module 204. When the dangerous action detection model detects that the movable target object in the in-vehicle environment image has dangerous behavior, the information generating module 204 is used to:
[0089] The behavior type corresponding to the dangerous behavior is determined based on the dangerous action detection model, and the second alarm information is generated based on the behavior type.
[0090] Furthermore, the multiple preset positions include a co-pilot position, a left side position of the rear seat, and a right side position of the rear seat.
[0091] Further, such as Figure 3 As shown, the in-vehicle environment monitoring device 200 further includes a locking instruction generating module 205. When a dangerous action monitoring instruction is received and it is detected that the vehicle is driving, the locking instruction generating module 205 is used to:
[0092] Determine a target camera that captures an image of the vehicle interior environment where a movable target object exists;
[0093] Determine the target window and the target door corresponding to the target camera, and generate a forced locking instruction for the target window and the target door;
[0094] The forced locking instruction is sent to a vehicle body control module, so that the vehicle body control module forces the target vehicle window and the target vehicle door to be locked.
[0095] Further, such as Figure 3 As shown, the in-vehicle environment monitoring device 200 further includes a first model training module 206, and the first model training module 206 is used to train the dangerous action detection model through the following steps:
[0096] Acquire multiple dangerous action sample images; wherein each dangerous action sample image has a corresponding dangerous action sample type;
[0097] Inputting a plurality of the dangerous action sample images into a dangerous action original detection model to determine a dangerous action prediction type corresponding to each dangerous action sample image;
[0098] Compare the dangerous action sample type corresponding to each dangerous action sample image with the dangerous action prediction type, and calculate the loss function of the original detection model of the dangerous action in the current state;
[0099] Based on the loss function of the original dangerous action detection model, the model parameters of the original dangerous action detection model are continuously adjusted until the original dangerous action detection model reaches a convergence state, thereby obtaining the dangerous action detection model.
[0100] Further, such as Figure 3 As shown, the in-vehicle environment monitoring device 200 further includes a second model training module 207, and the second model training module 207 is used to train the object detection model through the following steps:
[0101] Acquire a plurality of object sample images; wherein the plurality of object sample images include a plurality of sample images containing movable objects and a plurality of sample images containing immovable objects, and each object sample image corresponds to a label value;
[0102] Inputting a plurality of the object sample images into an original object detection model to determine a prediction value corresponding to each object sample image;
[0103] Compare the label value corresponding to each object sample image with the predicted value, and calculate the loss function of the original detection model of the object in the current state;
[0104] Based on the loss function of the original object detection model, the model parameters of the original object detection model are continuously adjusted until the original object detection model reaches a convergence state, thereby obtaining the object detection model.
[0105] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in , the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .
[0106] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 The specific implementation of the steps of the in-vehicle environment monitoring method in the method embodiment shown can be found in the method embodiment, and will not be repeated here.
[0107] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The specific implementation of the steps of the in-vehicle environment monitoring method in the method embodiment shown can be found in the method embodiment, and will not be repeated here.
[0108] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0109] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0110] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0111] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0112] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0113] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A method for monitoring the in-vehicle environment, characterized in that: The in-vehicle environment monitoring method comprises: For each camera installed at a plurality of preset positions in the vehicle, an image of the vehicle interior environment captured by the camera is obtained; Inputting the in-vehicle environment image into an object detection model, and generating prompt information when the object detection model detects that there is a movable target object in the in-vehicle environment image; wherein the movable target object is a child or a pet; When a dangerous action monitoring instruction is received and it is detected that the vehicle is moving, the interior environment image captured by the camera is input into the dangerous action detection model. When the dangerous action detection model detects that a movable target object in the interior environment image has dangerous behavior, a first alarm message is generated.
2. The method for monitoring the in-vehicle environment according to claim 1, characterized in that: When the dangerous action detection model detects that the movable target object in the in-vehicle environment image has dangerous behavior, the in-vehicle environment monitoring method further includes: The behavior type corresponding to the dangerous behavior is determined based on the dangerous action detection model, and the second alarm information is generated based on the behavior type.
3. The method for monitoring the in-vehicle environment according to claim 1, characterized in that: The multiple preset positions include a co-pilot position, a left rear seat position, and a right rear seat position.
4. The method for monitoring the in-vehicle environment according to claim 3, characterized in that: When a dangerous action monitoring instruction is received and it is detected that the vehicle is driving, the in-vehicle environment monitoring method further includes: Determine a target camera that captures an image of the vehicle interior environment where a movable target object exists; Determine the target window and the target door corresponding to the target camera, and generate a forced locking instruction for the target window and the target door; The forced locking instruction is sent to a vehicle body control module, so that the vehicle body control module forces the target vehicle window and the target vehicle door to be locked.
5. The method for monitoring the in-vehicle environment according to claim 2, characterized in that: The dangerous action detection model is trained by the following steps: Acquire multiple dangerous action sample images; wherein each dangerous action sample image has a corresponding dangerous action sample type; Inputting a plurality of the dangerous action sample images into a dangerous action original detection model to determine a dangerous action prediction type corresponding to each dangerous action sample image; Compare the dangerous action sample type corresponding to each dangerous action sample image with the dangerous action prediction type, and calculate the loss function of the original detection model of the dangerous action in the current state; Based on the loss function of the original dangerous action detection model, the model parameters of the original dangerous action detection model are continuously adjusted until the original dangerous action detection model reaches a convergence state, thereby obtaining the dangerous action detection model.
6. The method for monitoring the in-vehicle environment according to claim 1, characterized in that: The object detection model is trained by the following steps: Acquire a plurality of object sample images; wherein the plurality of object sample images include a plurality of sample images containing movable objects and a plurality of sample images containing immovable objects, and each object sample image corresponds to a label value; Inputting a plurality of the object sample images into an original object detection model to determine a prediction value corresponding to each object sample image; Compare the label value corresponding to each object sample image with the predicted value, and calculate the loss function of the original detection model of the object in the current state; Based on the loss function of the original object detection model, the model parameters of the original object detection model are continuously adjusted until the original object detection model reaches a convergence state, thereby obtaining the object detection model.
7. A vehicle interior environment monitoring device, characterized in that: The in-vehicle environment monitoring device comprises: An image acquisition module, for acquiring an image of the vehicle interior environment captured by each camera installed at a plurality of preset positions in the vehicle; A first detection module is used to input the in-vehicle environment image into an object detection model, and generate prompt information when the object detection model detects that there is a movable target object in the in-vehicle environment image; wherein the movable target object is a child or a pet; The second detection module is used to input the interior environment image captured by the camera into the dangerous action detection model when a dangerous action monitoring instruction is received and it is detected that the vehicle is moving, and generate a first alarm message when the dangerous action detection model detects that a movable target object in the interior environment image has dangerous behavior.
8. The in-vehicle environment monitoring device according to claim 7, characterized in that: The in-vehicle environment monitoring device further includes an information generation module, and when the dangerous action detection model detects that the movable target object in the in-vehicle environment image has dangerous behavior, the information generation module is used to: The behavior type corresponding to the dangerous behavior is determined based on the dangerous action detection model, and the second alarm information is generated based on the behavior type.
9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the in-vehicle environment monitoring method as described in any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the in-vehicle environment monitoring method as claimed in any one of claims 1 to 6 are executed.