An in-vehicle abnormal object detection processing method and device

Abstract

The application discloses a kind of in-vehicle abnormal object detection processing method and device.The in-vehicle abnormal object detection processing method includes: by at least three cameras deployed in vehicle, the continuous image of the abnormal object entering vehicle is acquired;The actual size of the abnormal object is determined based on the image of the same moment photographed by different cameras;The motion trajectory of the abnormal object is determined based on the multiple frames of continuous images photographed by the same camera;According to the predetermined processing mode, different sizes, different motion trajectories or multiple foreign matter invasion situations combined by different sizes and different motion trajectories, the locomotive control system drives the locomotive to make corresponding action.The application can improve the practicability of intrusion detection, improve the safety of vehicle, and also improve the comfort during driving.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of vehicle automatic control technology, in particular to an abnormal object detection processing method and device in a vehicle. BACKGROUND

[0002] When driving a vehicle, or temporarily leaving the vehicle, the driver often does not close the window. When there is a need to open the window, especially in summer, some foreign objects will inevitably float into the window, especially flies and other flying insects, which will particularly affect the driving mood. Or when the car owner parks the car, forgets or does not timely close the window, resulting in theft of the car's property. Or forget to close the car window on a rainy day, resulting in the soaking of the floor mat, carpet and seat, which will cause decay, shorten the service life and produce odor.

[0003] In the prior art, the foreign object entering the vehicle due to the open window is usually determined by detection technology, and the driver is prompted to take appropriate action. However, the existing vehicle intrusion detection technology mainly focuses on how to detect intrusion, without distinguishing the object that intrudes. The probability of false detection is also high. Therefore, even if the intrusion is detected, timely and effective countermeasures cannot be taken.

[0004] Therefore, it is desirable to have a technical solution to solve or at least alleviate the above-mentioned deficiencies of the prior art. SUMMARY

[0005] The purpose of the present application is to provide an abnormal object detection processing method and device in a vehicle to at least solve one of the above technical problems.

[0006] The first aspect of the present application provides an abnormal object detection processing method in a vehicle, mainly comprising:

[0007] obtaining continuous images of the abnormal object entering the vehicle through at least three cameras deployed in the vehicle;

[0008] determining the actual size of the abnormal object based on the images of the same moment taken by different cameras, and dividing the abnormal object into large objects exceeding a size threshold and small objects opposite to the large objects based on the actual size of the abnormal object;

[0009] determining the motion trajectory of the abnormal object based on multiple continuous images taken by the same camera, and dividing the abnormal object into non-living things in the form of free fall and living things opposite to the non-living things based on the motion trajectory of the abnormal object;

[0010] According to the predetermined processing mode, different sizes, different motion trajectories or multiple foreign object intrusion situations combined by different sizes and different motion trajectories are driven by the locomotive control system to make corresponding actions.

[0011] Preferably, the determining the actual size of the abnormal object comprises:

[0012] acquiring position information of the abnormal object entering the vehicle based on at least three groups of cameras deployed in the vehicle, and determining the distance between the object and the cameras based on distance sensors installed at the cameras;

[0013] determining the actual size of the abnormal object based on the distance between the abnormal object and the cameras and the imaging size of the abnormal object under the cameras.

[0014] Preferably, the determining the actual size of the abnormal object based on the distance between the abnormal object and the cameras and the imaging size of the abnormal object under the cameras comprises:

[0015] respectively acquiring the imaging size of the abnormal object and the distance between the abnormal object and each camera photographed by the three cameras;

[0016] respectively determining the actual size of the abnormal object calculated by each of the three cameras;

[0017] performing weighted average on the three actual sizes to calculate the final actual size of the abnormal object.

[0018] Preferably, each of the three cameras calculates the actual size of the abnormal object by the following formula:

[0019] f = wL / W;

[0020] wherein f is the focal length of the camera, w is the imaging size of the abnormal object under the camera, L is the distance between the abnormal object and the camera, and W is the actual size of the abnormal object.

[0021] Preferably, the performing weighted average on the three actual sizes comprises:

[0022] determining the weighting coefficient of the actual size corresponding to each camera according to the imaging resolution of the camera.

[0023] Preferably, the driving the locomotive to make corresponding actions by the locomotive control system comprises:

[0024] when the abnormal object is a small object or a living object, triggering the vehicle pest control device to work by the locomotive control system;

[0025] when the abnormal object is a small object or a non-living object, performing the pre-operation of closing the vehicle window by the locomotive control system;

[0026] when the abnormal object is a large object, triggering the vehicle alarm device to perform the alarm operation by the locomotive control system.

[0027] Preferably, the pre-operation of closing the window performed by the locomotive control system comprises:

[0028] Determining whether there is a user in the vehicle, if there is a user in the vehicle, prompting the user to close the window, if there is no user in the vehicle, determining whether to drive the corresponding mechanical structure to close the window according to the preset closing window permission.

[0029] Preferably, when the locomotive control system triggers the alarm device in the vehicle to perform the alarm operation, it further comprises:

[0030] Sending the real-time image in the vehicle to the vehicle owner.

[0031] Preferably, before the continuous images of the abnormal object entering the vehicle are acquired by the at least three cameras deployed in the vehicle, it further comprises:

[0032] Based on the photoelectric sensor deployed at the window, it is detected whether there is an abnormal object passing through, if there is an abnormal object passing through, the camera is controlled to enter the working state, otherwise the camera is in standby state.

[0033] The second aspect of the present application provides an abnormal object detection processing device in a vehicle, mainly comprising:

[0034] An image acquisition module for acquiring continuous images of an abnormal object entering the vehicle through at least three cameras deployed in the vehicle;

[0035] An abnormal object size determination module for determining the actual size of the abnormal object based on images taken by different cameras at the same time, and dividing the abnormal object into large objects exceeding a size threshold and small objects opposite to the large objects based on the actual size of the abnormal object;

[0036] An abnormal object type determination module for determining the motion trajectory of the abnormal object based on multiple continuous images taken by the same camera, and dividing the abnormal object into non-living things in the form of free fall and living things opposite to the non-living things based on the motion trajectory of the abnormal object;

[0037] A locomotive automatic processing module for driving the locomotive to make corresponding actions according to a predetermined processing mode for different sizes, different motion trajectories, or multiple abnormal object intrusion situations combined by different sizes and different motion trajectories.

[0038] Preferably, the abnormal object size determination module comprises:

[0039] A position and distance acquisition unit for acquiring position information of an abnormal object entering the vehicle based on at least three groups of cameras deployed in the vehicle, and determining the distance between the object and the camera based on the distance sensor installed at the camera;

[0040] an actual size calculation unit configured to determine an actual size of the abnormal object based on the distance between the abnormal object and the camera and the imaging size of the abnormal object under the camera.

[0041] Preferably, the actual size calculation unit comprises:

[0042] a single-camera data acquisition subunit configured to acquire the imaging size of the abnormal object under each of the three cameras and the distance between the abnormal object and each of the three cameras, respectively;

[0043] a single-camera actual size calculation subunit configured to determine the actual size of the abnormal object calculated by each of the three cameras, respectively;

[0044] an actual size weighted calculation subunit configured to perform weighted averaging on the three actual sizes to calculate a final actual size of the abnormal object.

[0045] Preferably, the single-camera actual size calculation subunit is equipped with the following size calculation model:

[0046] f = wL / W;

[0047] wherein f is the focal length of the camera, w is the imaging size of the abnormal object under the camera, L is the distance between the abnormal object and the camera, and W is the actual size of the abnormal object.

[0048] Preferably, the actual size weighted calculation subunit comprises a weighting coefficient determination model configured to determine the weighting coefficient of the actual size corresponding to each camera according to the imaging resolution of the camera.

[0049] Preferably, the locomotive automatic processing module comprises:

[0050] an insect expelling unit configured to trigger the operation of an insect expelling device in the locomotive by the locomotive control system when the abnormal object is a small object and a living object;

[0051] a window operating unit configured to perform a pre-set operation of closing the window by the locomotive control system when the abnormal object is a small object and a non-living object;

[0052] an alarm unit configured to trigger the operation of an alarm device in the locomotive by the locomotive control system when the abnormal object is a large object.

[0053] Preferably, the window operating unit comprises:

[0054] a user identification unit configured to determine whether there is a user in the locomotive, and if there is a user in the locomotive, to issue a prompt for the user to close the window, and if there is no user in the locomotive, to determine whether to drive the corresponding mechanical structure to close the window according to a pre-set closing window permission.

[0055] Preferably, the alarm unit further comprises:

[0056] An image sending unit is configured to send the real-time image in the vehicle to the owner.

[0057] Preferably, the abnormal object detection processing device further comprises:

[0058] The foreign matter intrusion starting module is configured to detect whether there is an abnormal object passing through based on the photoelectric sensor arranged at the window, and if there is an abnormal object passing through, control the camera to enter the working state, otherwise, make the camera in standby state.

[0059] The third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the above-mentioned abnormal object detection processing method in the vehicle when executing the computer program.

[0060] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program can implement the above-mentioned abnormal object detection processing method in the vehicle when executed by a processor.

[0061] Advantages:

[0062] The present application can identify the objects intruding into the vehicle, classify them, and cooperate with other devices to implement timely and effective measures; the present application uses multiple devices to work together, which can greatly reduce the possibility of false detection; the present application can improve the practicability of intrusion detection, improve the safety of the vehicle, and also improve the comfort during driving. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 is a flow chart of the abnormal object detection processing method in the vehicle according to an embodiment of the present application.

[0064] Figure 2 is a flow chart of the abnormal object detection processing method in the vehicle according to another embodiment of the present application.

[0065] Figure 3 is a schematic diagram of an electronic device capable of implementing the abnormal object detection processing method in the vehicle according to an embodiment of the present application. DETAILED DESCRIPTION

[0066] For the purpose of the embodiments of the present application, the technical solutions and advantages, the technical solutions of the embodiments of the present application will be described in more detail below with reference to the drawings in the embodiments of the present application. In the drawings, the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The described embodiments are part of the embodiments of the present application, not all embodiments. The embodiments described below by reference to the drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application. The embodiments of the present application will be described in detail below with reference to the drawings.

[0067] Figure 1 FIG. 1 is a flowchart of a vehicle abnormal object detection processing method according to an embodiment of the present application.

[0068] As shown in FIG. 1, the vehicle abnormal object detection processing method includes the following steps. Figure 1

[0069] Step S1, acquiring continuous images of an abnormal object entering the vehicle through at least three cameras deployed in the vehicle;

[0070] Step S2, determining the actual size of the abnormal object based on images of the same moment taken by different cameras, and dividing the abnormal object into large objects exceeding a size threshold and small objects opposite to the large objects based on the actual size of the abnormal object;

[0071] Step S3, determining the motion trajectory of the abnormal object based on multiple continuous images taken by the same camera, and dividing the abnormal object into non-living things in the form of free fall and living things opposite to the non-living things based on the motion trajectory of the abnormal object;

[0072] Step S4, driving the locomotive to make corresponding actions by the locomotive control system according to the predetermined processing mode for different sizes, different motion trajectories, or multiple foreign object intrusion situations combined by different sizes and different motion trajectories.

[0073] In some optional embodiments, the determination of the actual size of the abnormal object includes:

[0074] acquiring position information of the abnormal object entering the vehicle based on at least three groups of cameras deployed in the vehicle, and simultaneously determining the distance between the object and the camera based on the distance sensor installed at the camera;

[0075] determining the actual size of the abnormal object based on the distance between the abnormal object and the camera and the imaging size of the abnormal object under the camera.

[0076] ​The application can capture and identify the distance of the object by deploying three groups of cameras and distance sensors in the vehicle, and can analyze the behavior of the object according to the captured image and the distance of the object. The three cameras and distance sensors can sample several times within a certain time, and the size of the object can be calculated according to the pixels occupied by the object, the distance between the camera and the object, and the focal length of the camera. The three distances can locate the accurate position of the object, and the motion trajectory of the object can be obtained according to the position change after several samplings. In an alternative embodiment, the distance sensor can not be used, and the position and size of the object can be determined only by the pictures taken by the three cameras. Specifically, a three-dimensional coordinate system is constructed by the three cameras installed at fixed positions, and then the specific spatial coordinates of the abnormal object can be calculated according to the position of the picture where the abnormal object is located, and the distance of the abnormal object from each camera can be determined accordingly.

[0077] In some alternative embodiments, determining the actual size of the abnormal object based on the distance of the abnormal object from the camera and the imaging size of the abnormal object under the camera includes:

[0078] Respectively acquiring the imaging size of the abnormal object taken by the three cameras and the distance of the abnormal object from each camera;

[0079] Respectively determining the actual size of the abnormal object calculated by each of the three cameras;

[0080] Weighted averaging the three actual sizes to calculate the final actual size of the abnormal object.

[0081] In this embodiment, the size of the abnormal object can be calculated by three cameras and distance sensors, and then the mean value of the three values can be calculated by weighting as the final actual size of the abnormal object. When performing weighted calculation, the weighting coefficients of the three values can be the same or different. For example, the imaging resolution of the camera can be used as a reference value to determine the weighting coefficient of the actual size corresponding thereto, or the distance of the abnormal object from each camera can be used as a reference value to determine the weighting coefficient of the actual size calculated by each camera.

[0082] In some alternative embodiments, the three cameras calculate the actual size of the abnormal object by the following formula:

[0083] f = wL / W;

[0084] Where f is the focal length of the camera, w is the imaging size of the abnormal object under the camera, L is the distance of the abnormal object from the camera, and W is the actual size of the abnormal object.

[0085] In some alternative embodiments, the corresponding action of the locomotive driven by the locomotive control system includes:

[0086] When the abnormal object is a small object and a living object, the train control system triggers the operation of the train insect repellent device;

[0087] When the abnormal object is a small object and a non-living object, the train control system performs the pre-operation of closing the train window;

[0088] When the abnormal object is a large object, the train control system triggers the alarm device in the train to perform the alarm operation.

[0089] In this embodiment, the abnormal object with an actual size greater than 2 cm is determined as a large object, otherwise as a small object, and the specific size can be adjusted according to the actual situation. In step S3, the motion trajectory of the object can be known according to the position change after several samplings, and when the motion trajectory of the object is obviously inconsistent with the free-fall motion rule, the object is determined as a living object, otherwise as a non-living object. In step S4, when the object is determined as a small object and a living object, which is generally an insect invasion, the train insect repellent device (such as an electric insect repellent liquid) is triggered, and the object is driven out of the train by adjusting the air direction of the air conditioner. When the sensor detects that the driving is successful, the whole system reenters the standby state. When the object is determined as a small object and a non-living object, which is generally raindrops, fallen leaves, and floating objects, indicating that the weather or environment outside the train is not good at this time, the train control system performs the pre-operation of closing the train window, for example, determines whether there is a user in the train, if there is a user in the train, prompts the user to close the train window, and if there is no user in the train, determines whether to drive the corresponding mechanical structure to close the train window according to the preset closing train window permission. When the object is determined as a large object, which is generally human, human arm, theft tool or animal invasion, the train alarm device can be triggered. In some optional embodiments, the train network equipment (such as Tbox) can also be used to send real-time images in the train to the owner, and the owner can take corresponding countermeasures according to the image information.

[0090] In some optional embodiments, before the continuous images of the abnormal object entering the train are acquired by the at least three cameras deployed in the train, the method further comprises:

[0091] detecting whether there is an abnormal object passing through based on the photoelectric sensor deployed at the train window, and if there is an abnormal object passing through, controlling the camera to enter the working state, otherwise making the camera in the standby state.

[0092] In this embodiment, the camera starts to work only when the photoelectric sensor detects that the object passes through, avoiding the useless work of the camera for a long time. In addition, the determination of the actual size of the abnormal object in step S2 and the determination of the motion trajectory of the abnormal object in step S3 can be performed synchronously or sequentially, for example, the sequential execution of steps S2 and S3 in the case of mosquito invasion is explained.

[0093] Reference Figure 2 When the mosquito flies into the car window, the photoelectric sensor located at the upper part of the car window can detect the passing object and transmit the signal to the three cameras deployed in the car, starting to shoot images and running object analysis algorithms; The three cameras will all collect static images in advance as comparison images, and when the mosquito enters, the newly shot images and the comparison images are compared to obtain the pixel points occupied by the mosquito in the image. According to the pixel point range, the imaging size can be converted, and since the focal length of the camera is known, the distance of the object from the camera can be obtained through the distance sensor, and according to the imaging principle, the actual size of the mosquito can be calculated. If the intruding object is a small object in the previous step, the three cameras continue to sample, and according to the positions of the three distance sensors and the three cameras, the position of the object can be accurately determined. After a certain period of continuous sampling, the motion trajectory of the object can be outlined. The motion trajectory of the mosquito obviously does not conform to the free fall (parabolic line); According to this, it is judged that the object is a living object. At this time, the device can remind the car owner, and can also link the mosquito repellent device (such as mosquito repellent liquid) in the car to start the expulsion; When the expulsion is successful, the photoelectric sensor on the car window will detect that an object has flown out, and then the three cameras in the car will again confirm whether there is any foreign object moving in the car. If no detection is made, it is considered that the expulsion is successful, and the device in the car enters a standby state, waiting for the next intrusion signal from the photoelectric sensor.

[0094] The application can identify, classify and take timely and effective measures on the intruding objects of the vehicle; The application can greatly reduce the possibility of false detection by using multiple devices to work together; The application can improve the practicability of intrusion detection, improve the safety of the vehicle, and also improve the comfort during driving.

[0095] The second aspect of the application provides an in-vehicle abnormal object detection processing device corresponding to the above method, mainly comprising:

[0096] An image acquisition module is configured to acquire continuous images of an abnormal object entering the vehicle through at least three cameras deployed in the vehicle;

[0097] An abnormal object size determination module is configured to determine the actual size of the abnormal object based on images of the same time taken by different cameras, and divide the abnormal object into a large object exceeding a size threshold and a small object opposite to the large object based on the actual size of the abnormal object;

[0098] An abnormal object type determination module is configured to determine the motion trajectory of the abnormal object based on multiple continuous images taken by the same camera, and divide the abnormal object into a non-living object in the form of free fall and a living object opposite to the non-living object based on the motion trajectory of the abnormal object;

[0099] The locomotive automatic processing module is used for driving the locomotive to make corresponding actions according to a predetermined processing mode under different sizes, different motion trajectories or a combination of different sizes and different motion trajectories.

[0100] In some optional embodiments, the abnormal object size determination module comprises:

[0101] A position and distance acquisition unit is configured to acquire position information of the abnormal object entering the vehicle based on at least three groups of cameras arranged in the vehicle, and determine the distance between the object and the cameras based on distance sensors installed at the cameras.

[0102] An actual size calculation unit is configured to determine the actual size of the abnormal object based on the distance between the abnormal object and the cameras and the imaging size of the abnormal object under the cameras.

[0103] In some optional embodiments, the actual size calculation unit comprises:

[0104] A single-camera data acquisition subunit is configured to acquire the imaging size of the abnormal object and the distance between the abnormal object and each camera photographed by the three cameras, respectively.

[0105] A single-camera actual size calculation subunit is configured to determine the actual size of the abnormal object calculated by each of the three cameras, respectively.

[0106] An actual size weighted calculation subunit is configured to perform weighted averaging on the three actual sizes to calculate the final actual size of the abnormal object.

[0107] In some optional embodiments, the single-camera actual size calculation subunit is equipped with the following size calculation model:

[0108] f=wL / W;

[0109] Wherein, f is the focal length of the camera, w is the imaging size of the abnormal object under the camera, L is the distance between the abnormal object and the camera, and W is the actual size of the abnormal object.

[0110] In some optional embodiments, the actual size weighted calculation subunit comprises a weighting coefficient determination model configured to determine the weighting coefficient of the actual size corresponding to each camera according to the imaging resolution of the camera.

[0111] In some optional embodiments, the locomotive automatic processing module comprises:

[0112] A pest driving unit is configured to trigger the operation of a pest driving device in the vehicle by the locomotive control system when the abnormal object is a small object or a living object.

[0113] a window operating unit configured to perform a pre-set operation of closing the window by the locomotive control system when the abnormal object is a small non-living object;

[0114] an alarm unit configured to trigger an alarm device in the vehicle to perform an alarm operation when the abnormal object is a large object.

[0115] In some optional embodiments, the window operating unit comprises:

[0116] a user identifying unit configured to determine whether there is a user in the vehicle, and if there is a user in the vehicle, prompt the user to close the window, and if there is no user in the vehicle, determine whether to drive a corresponding mechanical structure to close the window according to a pre-set permission of closing the window.

[0117] In some optional embodiments, the alarm unit further comprises:

[0118] an image sending unit configured to send a real-time image in the vehicle to the vehicle owner.

[0119] In some optional embodiments, the abnormal object detecting processing device further comprises:

[0120] an intrusion starting module configured to detect whether there is an abnormal object passing through based on a photoelectric sensor deployed at the window, control the camera to enter a working state if there is an abnormal object passing through, and otherwise, control the camera to be in a standby state.

[0121] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the above abnormal object detecting processing method when executing the computer program.

[0122] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above abnormal object detecting processing method.

[0123] Figure 3 is an exemplary structural diagram of an electronic device capable of implementing the above abnormal object detecting processing method according to an embodiment of the application.

[0124] As Figure 3As shown, the electronic device includes an input device 501, an input interface 502, a central processor 503, a memory 504, an output interface 505, and an output device 506. Among them, the input interface 502, the central processor 503, the memory 504, and the output interface 505 are connected to each other through a bus 507, and the input device 501 and the output device 506 are connected to the bus 507 through the input interface 502 and the output interface 505 respectively, and then connected to other components of the electronic device. Specifically, the input device 504 receives input information from the outside, and transmits the input information to the central processor 503 through the input interface 502; the central processor 503 processes the input information based on the computer executable instructions stored in the memory 504 to generate output information, temporarily or permanently stores the output information in the memory 504, and then transmits the output information to the output device 506 through the output interface 505; the output device 506 outputs the output information to the outside of the electronic device for the user to use.

[0125] That is, Figure 3 The electronic device shown can also be implemented to include a memory storing computer executable instructions; and one or more processors which, when executing the computer executable instructions, can implement the in-vehicle abnormal object detection processing method described in combination with Figure 1 The description of the in-vehicle abnormal object detection processing method.

[0126] In one embodiment, Figure 3 The electronic device shown can be implemented to include a memory 504 configured to store executable program codes, and one or more processors 503 configured to run the executable program codes stored in the memory 504 to execute the in-vehicle abnormal object detection processing method in the above-described embodiments.

[0127] In one typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0128] The memory can include non-persistent memory in computer readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer readable media.

[0129] Computer-readable media includes permanent and non-permanent, movable and non-movable, media can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0130] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.

[0131] In addition, it is clear that the word "comprising" does not exclude other units or steps. Multiple units, modules or devices stated in device claims can also be provided by one unit or a total device, by means of software or hardware.

[0132] The flow diagrams and block diagrams in the drawings are schematic illustrations of possible architectures, functions and operations of systems, methods and computer program products in accordance with various embodiments of the present application. In this regard, each block in the flow diagrams and block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0133] The processor in this embodiment can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0134] The memory can be configured to store computer programs and / or modules, and the processor can be configured to implement various functions of the apparatus / terminal device by running or executing the computer programs and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, where the program storage area can store operating systems, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to use of the mobile phone (such as audio data, a phone book, etc.), and the like. In addition, the memory can include a high-speed random access memory, and can also include a nonvolatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash storage device, or other volatile solid-state storage devices.

[0135] In this embodiment, the modules / units integrated in the device / terminal equipment, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the computer program can be used to instruct the related hardware to complete all or part of the processes in the above-mentioned embodiment methods. The computer program can be stored in a computer readable storage medium, and the computer program can realize the steps of the above-mentioned method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. Although the above disclosure discloses the preferred embodiments, it is not intended to limit the application. Any possible changes and modifications made by those skilled in the art without departing from the spirit and scope of the application should be included in the protection scope of the application. The protection scope of the application should be subject to the scope defined by the claims of the application.

[0136] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the application can be embodied in a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage media, etc.) having computer usable program code embodied thereon.

[0137] In addition, it is clear that the word "comprising" does not exclude other units or steps. The units, modules or devices stated in the device claims can also be implemented by one unit or a general device by means of software or hardware.

[0138] Although the application has been described in detail above with specific reference to the preferred embodiments, it will be understood by those skilled in the art that various changes and modifications can be made without departing from the spirit and scope of the application. Therefore, the modifications and improvements made on the basis of the application should be included in the protection scope of the application.

Claims

1. A method for detecting and processing abnormal objects inside a vehicle, characterized in that, The method for detecting and processing abnormal objects inside the vehicle includes: At least three cameras deployed inside the vehicle acquire continuous images of any unusual object entering the vehicle. The actual size of the abnormal object is determined based on images taken at the same time by different cameras, and the abnormal object is divided into large objects exceeding a size threshold and relatively small objects based on the actual size of the abnormal object. The motion trajectory of the abnormal object is determined based on multiple consecutive images captured by the same camera, and the abnormal object is divided into non-living objects in the form of free fall and living objects in the opposite form based on the motion trajectory of the abnormal object. According to the predetermined processing method, the locomotive control system drives the locomotive to perform corresponding actions for various foreign object intrusion scenarios of different sizes, different movement trajectories, or combinations of different sizes and different movement trajectories; Determining the actual size of the abnormal object includes: The location information of the abnormal object entering the vehicle is obtained by at least three sets of cameras deployed inside the vehicle, and the distance between the object and the camera is determined by distance sensors installed at the cameras. The actual size of the abnormal object is determined based on the distance between the abnormal object and the camera, as well as the image size of the abnormal object under the camera. Determining the actual size of the anomalous object based on its distance from the camera and the image size of the anomalous object as seen by the camera includes: The image size of the abnormal object captured by the three cameras and the distance between the abnormal object and each camera are obtained respectively. The actual size of the abnormal object calculated by each of the three cameras is determined separately; The final actual size of the abnormal object is calculated by taking a weighted average of the three actual sizes. The three cameras each calculate the actual size of the anomalous object using the following formula: f = wL / W; Where f is the focal length of the camera, w is the image size of the abnormal object under the camera, L is the distance of the abnormal object from the camera, and W is the actual size of the abnormal object. The weighted average of the three actual dimensions includes: The weighting coefficients for the actual size corresponding to the image resolution of each camera are determined.

2. The method for detecting and processing abnormal objects inside a vehicle as described in claim 1, characterized in that, The actions performed by the locomotive control system include: When the abnormal object is a small, living creature, the locomotive control system triggers the in-vehicle insect repellent device to operate. When the abnormal object is a small, non-living object, the locomotive control system will perform the pre-operation of closing the window. When the abnormal object is a large object, the locomotive control system will trigger the in-vehicle alarm device to perform an alarm operation.

3. The method for detecting and processing abnormal objects inside a vehicle as described in claim 2, characterized in that, The pre-operation of closing the windows by the locomotive control system includes: It determines whether there is a user inside the vehicle. If there is a user, it prompts the user to close the window. If there is no user, it determines whether to activate the corresponding mechanical structure to close the window based on the preset window closing permissions.

4. The method for detecting and processing abnormal objects inside a vehicle as described in claim 3, characterized in that, When the locomotive control system triggers the in-vehicle alarm device to perform an alarm operation, it further includes: Send real-time images of the vehicle's interior to the owner.

5. The method for detecting and processing abnormal objects inside a vehicle as described in claim 1, characterized in that, Prior to acquiring continuous images of an unusual object entering the vehicle using at least three cameras deployed inside the vehicle, the process further includes: The system uses photoelectric sensors deployed on the car windows to detect whether any abnormal objects are passing by. If an abnormal object is detected, the camera is activated and put into operation; otherwise, the camera is put into standby mode.

6. A device for detecting and processing abnormal objects inside a vehicle, characterized in that, The in-vehicle abnormal object detection and processing device includes: An image acquisition module is used to acquire continuous images of abnormal objects entering the vehicle using at least three cameras deployed inside the vehicle; An abnormal object size determination module is used to determine the actual size of the abnormal object based on images taken by different cameras at the same time, and to classify the abnormal object into large objects exceeding a size threshold and relatively small objects based on the actual size of the abnormal object. An abnormal object type determination module is used to determine the motion trajectory of the abnormal object based on multiple consecutive images captured by the same camera, and to classify the abnormal object into non-living objects in the form of free fall and living objects in the form of opposites based on the motion trajectory of the abnormal object. The locomotive automatic handling module is used to handle various foreign object intrusion scenarios of different sizes, different motion trajectories, or combinations of different sizes and different motion trajectories according to a predetermined handling method, and the locomotive control system drives the locomotive to perform corresponding actions. Determining the actual size of the abnormal object includes: The location information of the abnormal object entering the vehicle is obtained by at least three sets of cameras deployed inside the vehicle, and the distance between the object and the camera is determined by distance sensors installed at the cameras. The actual size of the abnormal object is determined based on the distance between the abnormal object and the camera, as well as the image size of the abnormal object under the camera. Determining the actual size of the anomalous object based on its distance from the camera and the image size of the anomalous object as seen by the camera includes: The image size of the abnormal object captured by the three cameras and the distance between the abnormal object and each camera are obtained respectively. The actual size of the abnormal object calculated by each of the three cameras is determined separately; The final actual size of the abnormal object is calculated by taking a weighted average of the three actual sizes. The three cameras each calculate the actual size of the anomalous object using the following formula: f = wL / W; Where f is the focal length of the camera, w is the image size of the abnormal object under the camera, L is the distance of the abnormal object from the camera, and W is the actual size of the abnormal object. The weighted average of the three actual dimensions includes: The weighting coefficients for the actual size corresponding to the image resolution of each camera are determined.

Images