In-vehicle leaving detection method, system and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of automobiles, and discloses an in-vehicle leaving detection method, system and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a first image containing the front seat space of a vehicle; performing leaving detection on the front-row seat space according to the first image, and executing vehicle control according to a detection result; under the condition that the target remnant object is not detected in the first image, the front-row seat is controlled to be subjected to posture adjustment, so that the adjusted front-row seat is far away from a shielding area of a rear-row seat of the vehicle; after the posture of the front-row seat is adjusted, a second image containing the rear-row seat space of the vehicle is obtained; and carrying out leaving detection on the rear-row seat space according to the second image, and executing vehicle control according to a detection result. According to the invention, the reliability of in-vehicle leaving-in detection can be improved.

Description

Technical Field

[0001] The present invention relates to the field of automobile technology, and in particular to a method, system, device, electronic equipment and storage medium for detecting in-vehicle residue. Background Art

[0002] With the rapid development of the automobile industry, more and more high-end models are equipped with in-car left-behind detection systems to prevent car owners from leaving important items in the car.

[0003] The current in-vehicle legacy detection system is usually based on a camera or millimeter-wave radar. However, both the camera field of view and the millimeter-wave penetration range will be blocked by objects in the car (such as the seat backs), thus forming a blind spot. This makes it impossible for the in-vehicle legacy detection to accurately provide the situation inside the car, resulting in low reliability of the in-vehicle legacy detection. Summary of the invention

[0004] Embodiments of the present invention provide a method, system, device, electronic device and storage medium for detecting items left in a vehicle, so as to solve the problem of low reliability of detecting items left in a vehicle.

[0005] A method for detecting residual items in a vehicle, the method comprising:

[0006] Acquiring a first image including a front seat space of a vehicle;

[0007] Performing a left-over detection on the front seat space according to the first image, and executing vehicle control according to the detection result;

[0008] When no target remains are detected in the first image, controlling the front seat to adjust its posture so that the adjusted front seat is away from the shielding area of ​​the rear seat of the vehicle;

[0009] After the front seat posture adjustment is completed, acquiring a second image including the rear seat space of the vehicle;

[0010] A carry-over detection is performed on the rear seat space according to the second image, and vehicle control is performed according to the detection result.

[0011] In the above method, optionally, performing a leftover detection on the rear seat space according to the second image and controlling the vehicle according to the detection result includes:

[0012] When no target object is detected in the second image, the front seat is restored to an initial posture.

[0013] In the above method, optionally, performing a leftover detection on the front seat space according to the first image and executing vehicle control according to the detection result includes:

[0014] identifying whether there are objects and / or living beings left in the front seat space according to the first image;

[0015] If the life form exists in the front seat space, executing an alarm strategy;

[0016] If the left object exists in the front seat space, a prompt strategy is executed.

[0017] In the above method, optionally, performing a leftover detection on the rear seat space according to the second image and executing vehicle control according to the detection result includes:

[0018] identifying whether there are objects and / or living beings left in the rear seat space according to the second image;

[0019] If the life form exists in the rear seat space, executing an alarm strategy;

[0020] If the left object exists in the rear seat space, a prompt strategy is executed.

[0021] In the above method, optionally, the vehicle further comprises a third row of seats, the front row of seats is located in front of the rear row of seats, and the rear row of seats is located in front of the third row of seats;

[0022] In the case where no target remains are detected in the second image, before restoring the front seat to an initial posture, the method further includes:

[0023] Controlling the rear seat to adjust its posture so that the adjusted rear seat is away from the shielding area of ​​the third row seat of the vehicle;

[0024] After the rear seat posture adjustment is completed, acquiring a third image including the third row seat space of the vehicle;

[0025] Performing a leftover detection on the third row seat space according to the third image, and controlling the vehicle according to the detection result;

[0026] In the case where the target object is detected in the third image, an alarm and / or prompt strategy is executed; in the case where the target object is not detected in the third image, the rear seat is restored to an initial posture.

[0027] In the above method, optionally, the alarm strategy includes:

[0028] Send an alarm message to the car owner's mobile terminal; or, call the user's alarm number through intelligent customer service; or, automatically open the vehicle's ventilation system;

[0029] The prompt strategy includes: sending a prompt message to the mobile terminal of the car owner.

[0030] In the above method, optionally, the front seats include a left front seat and a right front seat;

[0031] The performing of a leftover detection on the front seat space according to the first image includes:

[0032] The left front seat space and the right front seat space are respectively subjected to a carry-over detection according to the first image.

[0033] Optionally, in the above method, controlling the posture adjustment of the front seat includes: adjusting the slide rail and backrest of the front seat to the frontmost position.

[0034] A vehicle leftover detection system, the system comprising: a camera, an intelligent driving domain controller, a central control unit,

[0035] The camera is used to obtain a first image including the front seat space of the vehicle;

[0036] The intelligent driving domain controller is used to receive a first image transmitted by a camera, and perform a legacy detection on the front seat space based on the first image;

[0037] The central control unit is configured to control the front seats to perform posture adjustment when the intelligent driving domain controller does not detect the target remains in the first image, so that the adjusted front seats are away from the shielding area of ​​the rear seats of the vehicle;

[0038] The camera is used to obtain a second image including the rear seat space of the vehicle through the camera after the front seat posture adjustment is completed;

[0039] The intelligent driving domain controller is used to receive the second image and perform legacy detection on the rear seat space based on the second image;

[0040] The central control unit is used to restore the front seats to their initial posture when the intelligent driving domain controller does not detect the target remains in the second image.

[0041] A detection device for residual items in a vehicle, comprising:

[0042] A first image acquisition unit, configured to acquire a first image including a front seat space of the vehicle through a camera;

[0043] a first left-behind detection unit, configured to input the first image into a recognition module to perform left-behind detection on the front seat space;

[0044] a first posture adjustment unit, configured to control the front seat to perform posture adjustment when no target object is detected in the front seat space, so that the adjusted front seat is away from a shielding area of ​​a rear seat of the vehicle;

[0045] a second image acquisition unit, configured to acquire, through the camera, a second image including a rear seat space of the vehicle after the front seat posture adjustment is completed;

[0046] a second left-behind detection unit, configured to input the second image into a recognition module to perform left-behind detection on the rear seat space;

[0047] The second posture adjustment unit is used to restore the front seat to an initial posture when no target object is detected in the rear seat space.

[0048] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned in-vehicle residual detection method when executing the computer program.

[0049] A readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the above-mentioned in-vehicle residual detection method.

[0050] The above-mentioned in-vehicle leftover detection method, system, device, electronic device and storage medium, the method first obtains a first image including the front seat space of the vehicle, performs leftover detection on the front seat space according to the first image, and controls the front seat to adjust its posture when no target leftover is detected in the front seat space, and then obtains a second image including the rear seat space of the vehicle, and performs leftover detection on the rear seat space according to the second image. It can be seen that in the present invention, the in-vehicle leftover detection is first performed on the front seat space, and then the posture of the front seat is adjusted, and then the in-vehicle leftover detection is performed on the rear seat space, thereby preventing the field of view of the shooting equipment from being blocked by objects in the vehicle during the in-vehicle leftover detection process, thereby forming a blind spot, and having high reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0052] Figure 1 It is a flow chart of a method for detecting in-vehicle residues disclosed in the first embodiment of the present invention;

[0053] Figure 2 This is a structural schematic diagram of a camera and a seat disclosed in the first embodiment of the present invention;

[0054] Figure 3 is another structural schematic diagram of a camera and a seat disclosed in the first embodiment of the present invention;

[0055] Figure 4 It is a partial flow chart of a method for detecting in-vehicle residues disclosed in the first embodiment of the present invention;

[0056] Figure 5 It is a partial flow chart of a method for detecting in-vehicle residues disclosed in the first embodiment of the present invention;

[0057] Figure 6 This is a schematic diagram of the distribution structure of left and right seats in a vehicle disclosed in the first embodiment of the present invention;

[0058] Figure 7 It is another structural schematic diagram of a camera and a seat disclosed in the first embodiment of the present invention;

[0059] Figure 8 is a structural schematic diagram of a vehicle left-behind detection system disclosed in Embodiment 2 of the present invention;

[0060] Fig. 9 is a control principle diagram of a vehicle left-behind detection system disclosed in Embodiment 3 of the present invention;

[0061] Fig.10 It is a structural schematic diagram of a vehicle leftover detection device disclosed in Embodiment 3 of the present invention;

[0062] Fig.11 It is a structural schematic diagram of an electronic device disclosed in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0065] It should also be understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0066] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]", depending on the context.

[0067] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0068] References to "one embodiment" or "some embodiments" etc. described in the present specification mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0069] In the above-mentioned in-vehicle residue detection method, system, device, electronic device and storage medium, a first image including the front seat space of the vehicle is first acquired, and a residue detection is performed on the front seat space according to the first image. When the target residue is not detected in the front seat space, the front seat is controlled to adjust its posture. Then, a second image including the rear seat space of the vehicle is acquired, and a residue detection is performed on the rear seat space according to the second image. When the target residue is not detected in the rear seat space, the front seat is restored to its initial posture. It can be seen that in the present invention, the in-vehicle residue detection is first performed on the front seat space, and then the posture of the front seat is adjusted, and then the in-vehicle residue detection is performed on the rear seat space. Therefore, it can be avoided that the field of view of the shooting equipment will be blocked by objects in the vehicle during the in-vehicle residue detection process, thereby forming a blind spot, and the reliability is high.

[0070] It should be understood that in this embodiment, the first image obtained only contains the front seat space, while the rear seat space is blocked by the backrest of the front seat. Because the rear seat space belongs to a blind spot of vision, after performing residue detection on the front seats according to the first image, if there is no target residue in the front seat space, the front seats are controlled to adjust their posture so that the backrest of the front seat is away from the blocked area of ​​the rear seat. Then, a second image is obtained, which includes the rear seat space. According to the second image, the rear seat is detected for residues, and if there is no target residue in the rear seat space, the front seats are controlled to restore their original posture.

[0071] In addition, the vehicle in the present invention may include but is not limited to a four-seat double-row vehicle, a six-seat three-row vehicle, etc. When the vehicle in the present invention is a four-seat double-row vehicle, the interior of the vehicle includes the above-mentioned front seats and rear seats, and the front seat space and rear seat space of the vehicle are respectively detected for legacy; when the vehicle in the present invention is a six-seat three-row vehicle, the interior of the vehicle includes the above-mentioned front seats and rear seats, and also includes a third row of seats, and the front seat space, rear seat space and third row seat space of the vehicle are respectively detected for legacy. The present invention does not specifically limit the number of rows and the number of seats in the vehicle.

[0072] In addition, as an example, after the driver leaves the vehicle and locks the vehicle, the vehicle is powered off with a delay, that is, when the vehicle is powered on, a left-behind detection is performed on the vehicle, and the vehicle is powered off when the left-behind detection is completed and it is confirmed that there are no left objects or living things in the vehicle.

[0073] Embodiment 1

[0074] refer to Figure 1 FIG. 1 is a flow chart of a method for detecting objects left in a vehicle disclosed in Embodiment 1 of the present invention. The method is suitable for alarming or prompting when objects left in a vehicle are present. Specifically, the method is as follows:

[0075] S101: Acquire a first image including a front seat space of a vehicle.

[0076] The first image can be obtained by a camera. The camera can be set near the rearview mirror. Figure 2 The figure shows a schematic diagram of the structure of the camera and the seat in the embodiment of the present invention. It can be seen that when the camera in the figure acquires the first image, the first image includes a complete image of the front seat space, while the rear seat space is blocked by the front seat, resulting in that the first image does not include a complete image of the rear seat space. Accordingly, the subsequent steps of the vehicle interior residual detection method are performed according to the obtained first image.

[0077] In addition, when the first image is acquired through the camera, the auxiliary lights in the vehicle can be adjusted to the highest brightness through the central control unit of the vehicle, and the auxiliary lights can be adjusted to daily lighting, thereby acquiring a high-quality first image.

[0078] S102: Perform a left-behind detection on the front seat space according to the first image, and execute vehicle control according to the detection result.

[0079] In a specific implementation, the first image can be input into a recognition module. The recognition module in this embodiment may include a pre-trained image recognition model. Residue detection is performed on the first image through the image recognition model. When target relics are detected in the first image, an alarm or prompt is given according to the type or category of the target relics. When no target relics are detected in the first image, step S103 is executed, i.e., the front seats are controlled to adjust their posture.

[0080] Among them, the camera collects the current in-vehicle environment picture for the first time and gives it to the recognition module for identification of relics and living things. The identification results can be divided into three categories: living things (regardless of whether there are relics or not), inanimate objects but with relics, and inanimate objects and relics.

[0081] In the case of the presence of living things, an alarm strategy needs to be adopted (the alarm has the highest priority, for example, the vehicle cloud sends an alarm text message to the owner's mobile phone at the first time, and sends a photo of the interior environment to the owner's mobile phone APP. If the owner does not cancel the alarm within three minutes, the smart customer service will call the owner's mobile phone to remind him. If the owner does not cancel the alarm within five minutes, the vehicle will actively turn on the air conditioner and sound the horn to alarm; the above alarm strategies are only examples, and the specific strategies can be changed according to design requirements). For details, please see the subsequent embodiments.

[0082] It should be noted that when the target object is detected in the first image, it means that the target object is present in the front seat space, so the front seat cannot be controlled to adjust its posture, and an alarm or prompt is issued according to the type or category of the target object.

[0083] S103: Control the front seats to adjust their posture.

[0084] In a specific implementation, in this embodiment, the vehicle's central control unit CCU can be used to control the vehicle seat to be automatically adjusted, such as controlling the slide rails and backrests of the vehicle's front seats to be adjusted to the front end, so that the adjusted front seats are away from the shielding area of ​​the vehicle's rear seats. Figure 3 FIG. 1 is a schematic diagram of the structure of the camera and the seat in an embodiment of the present invention. It can be seen that the adjusted front seats are away from the shielding area of ​​the rear seats, so that the second image obtained by the camera includes a complete image of the rear seat space, thereby performing a residual detection on the rear seat space.

[0085] S104: After the front seat posture adjustment is completed, a second image including the rear seat space of the vehicle is acquired.

[0086] refer to Figure 3 As shown, the acquired second image contains a complete image of the rear seat space, and the subsequent steps of the vehicle interior residue detection method are performed according to the acquired first image.

[0087] S105: Perform a left-behind detection on the rear seat space according to the second image, and execute vehicle control according to the detection result.

[0088] In a specific implementation, the second image can be input into a recognition module. The recognition module in this embodiment may include a pre-trained image recognition model. Residue detection is performed on the second image through the image recognition model. When the target relics are detected in the second image, an alarm or prompt is given according to the type or category of the target relics. When the target relics are not detected in the second image, step S106 is executed to restore the front seat to its initial posture.

[0089] S106: Restore the front seats to their initial positions.

[0090] The initial state may be the posture of the seat rails and the posture of the seat back before the posture of the front seat of the vehicle is adjusted, or the initial state may be the posture of the seat rails and the posture of the seat back preset for the front seat of the vehicle. Restoring the front seat to the initial posture is convenient for the user to use the vehicle next time.

[0091] In addition, when the present invention performs the above-mentioned in-car leftover detection method, the in-car light (such as in-car reading light) can be turned on to enable the camera to collect images and obtain clearer pictures to improve the recognition accuracy of the recognition module. In addition, the seat in this embodiment should have an anti-pinch function to avoid pinching life forms and leftover objects in extreme cases, and the specific anti-pinch force can be calibrated according to needs.

[0092] In summary, the first embodiment of the present invention discloses a method for detecting objects left behind in a vehicle, which method first acquires a first image of the front seat space of the vehicle, inputs the first image into a recognition module, performs object left behind detection on the front seat space, and controls the front seats to adjust their posture when no target object left behind is detected in the front seat space, and then acquires a second image of the rear seat space of the vehicle, inputs the second image into a recognition module, performs object left behind detection on the rear seat space, and restores the front seats to their initial posture when no target object left behind is detected in the rear seat space. It can be seen that in the present invention, object left behind detection is first performed on the front seat space, and then the front seats are adjusted in posture, and then the rear seat space is performed on object left behind detection, thereby avoiding the situation where the field of view of the shooting equipment is blocked by objects in the vehicle during the object left behind detection process, thereby forming a blind spot, and the method has high reliability.

[0093] based on Figure 1 In the specific implementation, step S102 can also be implemented by the following steps: Figure 4 As shown:

[0094] S401: Identify whether there are any objects and / or living beings left in the front seat space based on the first image.

[0095] If it is identified that there are objects left behind in the front seat space, execute step S402, i.e. execute the alarm strategy; if it is identified that there are living beings in the front seat space, execute step S403, i.e. execute the prompt strategy; if it is identified that there are no objects or living beings left behind in the front seat space, control the front seats to adjust their posture, and the adjusted front seats are away from the blocked area of ​​the rear seats of the vehicle.

[0096] In a specific implementation, the recognition module in this embodiment may include a pre-trained image recognition model, and the image recognition model is used to perform a leftover detection in the first image. When a life form is detected in the first image, an alarm strategy is executed; when a leftover object is detected in the first image, a prompt strategy is executed; when no leftover object or life form is detected in the first image, the front seat is controlled to adjust its posture. The leftover object may be an item carried by the user, such as a wallet, a folder, etc., and the life form may be a child, a pet (such as a cat, a dog, etc.). In this embodiment, there is no specific limitation on the leftover object and the life form.

[0097] For example, taking a family car as an example, when the user drives the vehicle into a parking space and locks the car with the car key, the camera installed in the car will take a picture of the interior of the car, and based on the captured pictures, the vehicle will be subjected to in-car medical testing. When the recognition module integrated in the vehicle's intelligent driving domain controller ADC detects the presence of items such as wallets and folders in the picture, the relevant alarm strategy or prompt strategy will be executed.

[0098] S402: Execute the alarm strategy.

[0099] When a life form is detected in the front seat space of the vehicle, the alarm strategy is executed. When the alarm strategy is executed, the front seat in the vehicle is no longer adjusted to ensure that the life form is not squeezed. The alarm strategy can be specifically as follows:

[0100] The first alarm strategy: sending an alarm message to the car owner's mobile terminal.

[0101] The alarm information at least includes the captured picture, and the mobile terminal may be a user's mobile phone, smart watch or other device.

[0102] For example, when a kitten is detected in a captured image, the vehicle intelligent driving domain controller ADC can send an alarm message carrying the image to the corresponding cloud server, and then send the alarm message to the user's mobile phone through the cloud server, so that the user can quickly understand the situation of the life left in the vehicle, so that the user can quickly return to the vehicle to avoid danger to the kitten in the car. Among them, the cloud server can send the alarm message to the user's mobile phone in the form of a text message to remind the user, or send the alarm message to the mobile phone software installed in the mobile phone that matches the vehicle, and send a message alarm through the mobile phone software to remind the user.

[0103] The second alarm strategy: call the user's alarm number through intelligent customer service.

[0104] For example, when a child is detected in the captured image, the vehicle automatically sends an alarm through the smart customer, notifying the police to quickly rescue the child in the car to prevent the child from suffocating if he or she stays in the car for a long time.

[0105] The third alarm strategy: automatically open the vehicle ventilation system.

[0106] For example, when a child is detected in the captured image, the vehicle ventilation system can be opened by the vehicle central control unit CCU to prevent the child from suffocating if he stays in the car for a long time.

[0107] It should be noted that when the presence of a life form in the vehicle is detected by the in-vehicle legacy detection method, any one or more of the above-mentioned execution strategies can be executed. For example, while sending an alarm message to the vehicle owner's mobile terminal, the vehicle ventilation system is automatically turned on, and within ten minutes after the alarm message is sent to the vehicle owner's mobile terminal, if the owner does not respond, the intelligent customer service will call the user's alarm number. In addition, in addition to the above three alarm strategies, other alarm strategies can also be included, such as controlling the vehicle to automatically turn on the double flash and honk the horn to remind passers-by to rescue the vehicle life form in time. Accordingly, the content of the alarm strategy is not specifically limited in this embodiment.

[0108] S403: Execute the prompt strategy.

[0109] When an object is detected to be left behind in the vehicle, the prompt strategy is executed. When the prompt strategy is executed, the front seats in the vehicle are no longer adjusted to ensure that the left-behind object will not be squeezed and damaged. The prompt strategy can be specifically as follows:

[0110] Send a prompt message to the car owner's mobile terminal, where the prompt information at least includes the captured picture, and the mobile terminal can be a user's mobile phone, smart watch or other device.

[0111] For example, when a wallet is detected in a captured image, the vehicle intelligent driving domain controller ADC can send a prompt message carrying the image to the corresponding cloud server, and then send the prompt message to the user's mobile phone through the cloud server, so that the user can quickly know that there is a wallet left in the vehicle, avoiding the user from blindly searching for the wallet when they need it. The cloud server can send the prompt message to the user's mobile phone in the form of a text message to remind the user, or send the prompt message to the mobile phone software installed in the mobile phone that matches the vehicle, and send a message reminder through the mobile phone software.

[0112] In addition, the seats in the vehicle in this embodiment may include not only two rows of seats, front seats and rear seats, but may also include three rows or even four rows of seats.

[0113] In some embodiments, the vehicle also includes a third row of seats, the front seats are located in front of the rear seats, and the rear seats are located in front of the third row of seats; when no target relics are detected in the second image, before the front seats are restored to their initial posture, it also includes: controlling the rear seats to adjust their posture so that the adjusted rear seats are away from the obstruction area of ​​the third row seats of the vehicle; after the rear seat posture adjustment is completed, obtaining a third image including the third row seat space of the vehicle; performing residue detection on the third row seat space according to the third image, and controlling the vehicle according to the detection results; when the target relics are detected in the third image, executing an alarm and / or prompt strategy; when no target relics are detected in the third image, restoring the rear seats to their initial posture.

[0114] As shown in Table 1, it is a schematic diagram of the detection results and prompt strategies when the vehicle has three rows of seats.

[0115] Table 1 Schematic diagram of the test results when the vehicle has three rows of seats

[0116]

[0117] It can be seen that when any row of seats detects the presence of objects left behind, the position of the seat where the objects are detected cannot be adjusted to avoid squeezing the objects left behind. Therefore, the rear seats of the seat cannot detect objects left behind. When the user is prompted for objects left behind, only the detected objects left behind and the row of seats that cannot be detected for objects left behind can be prompted. The prompt content can be as follows: "There are objects left behind in the first row of seats, and the second and third row of seats cannot be detected. Attached is a picture of the first row of seats." When no objects are detected on any row of seats in the vehicle, no prompt or alarm is given, and the seat position is restored to the initial position, which is convenient for the user to use the vehicle next time.

[0118] In summary, in this embodiment, a variety of alarm strategies and prompt strategies can be used to remind users of objects left in the car, so that users can promptly understand whether there are any objects left in the car, and can effectively ensure the safety of living things when there are living things in the car.

[0119] based on Figure 1 In the specific implementation, step S105 can be implemented by the following steps: Figure 5 As shown:

[0120] S501: Identify whether there are any objects and / or living beings left behind in the rear seat space based on the second image.

[0121] If it is recognized that there are objects left in the rear seat space, step S502 is executed, that is, the alarm strategy is executed; if it is recognized that there are living beings in the rear seat space, step S503 is executed, that is, the prompt strategy is executed; if it is recognized that there are no objects or living beings left in the rear seat space, the front seats are controlled to return to their initial posture.

[0122] In a specific implementation, the recognition module in this embodiment may include a pre-trained image recognition model, and the image recognition model is used to perform a leftover detection in the second image. When a life form is detected in the second image, an alarm strategy is executed; when a leftover object is detected in the second image, a prompt strategy is executed; when no leftover object or life form is detected in the second image, the front seat is controlled to return to the initial posture. Among them, the leftover object can be an item carried by the user, such as a wallet, a folder, etc., and the life form can be a child, a pet (such as a cat, a dog, etc.). In this embodiment, there is no specific limitation on the leftover object and the life form.

[0123] It should be noted that when it is detected that there are objects or living beings left behind in the rear seat space, the front seats will not return to their initial position to prevent the front seats from squeezing the objects or living beings left behind, effectively ensuring the safety of the objects or target living beings left behind.

[0124] S502: Execute the alarm strategy.

[0125] When a life form is detected in the rear seat space of the vehicle, the alarm strategy is executed. When the alarm strategy is executed, the front seats in the vehicle are no longer adjusted, that is, they are no longer restored to their initial positions to ensure that the life form is not squeezed. The specific alarm strategy can be as follows:

[0126] The first alarm strategy: sending an alarm message to the car owner's mobile terminal.

[0127] The alarm information at least includes the captured picture, and the mobile terminal may be a user's mobile phone, smart watch or other device.

[0128] For example, when a kitten is detected in a captured image, the vehicle intelligent driving domain controller ADC can send an alarm message carrying the image to the corresponding cloud server, and then send the alarm message to the user's mobile phone through the cloud server, so that the user can quickly understand the situation of the life left in the vehicle, so that the user can quickly return to the vehicle to avoid danger to the kitten in the car. Among them, the cloud server can send the alarm message to the user's mobile phone in the form of a text message to remind the user, or send the alarm message to the mobile phone software installed in the mobile phone that matches the vehicle, and send a message alarm through the mobile phone software to remind the user.

[0129] The second alarm strategy: call the user's alarm number through intelligent customer service.

[0130] For example, when a child is detected in the captured image, the vehicle automatically sends an alarm through the smart customer, notifying the police to quickly rescue the child in the car to prevent the child from suffocating if he or she stays in the car for a long time.

[0131] The third alarm strategy: automatically open the vehicle ventilation system.

[0132] For example, when a child is detected in the captured image, the vehicle ventilation system can be opened by the vehicle CCU central control unit to prevent the child from suffocating if he stays in the car for a long time.

[0133] It should be noted that when the presence of a living being in the vehicle is detected by the in-vehicle legacy detection method, any one or more of the above-mentioned execution strategies may be executed, and the content of the alarm strategy is not specifically limited in this embodiment.

[0134] S503: Execute the prompt strategy.

[0135] When an object is detected to be left behind in the vehicle, the prompt strategy is executed. When the prompt strategy is executed, the front seats in the vehicle are no longer adjusted to ensure that the left-behind object will not be squeezed and damaged. The prompt strategy can be specifically as follows:

[0136] Send a prompt message to the car owner's mobile terminal, where the prompt information at least includes the captured picture, and the mobile terminal can be a user's mobile phone, smart watch or other device.

[0137] For example, when a wallet is detected in a captured image, the vehicle ADC intelligent driving domain controller can send a prompt message with the image to the corresponding cloud server, and then send the prompt message to the user's mobile phone through the cloud server, so that the user can quickly know that there is a wallet left in the vehicle, avoiding the user from blindly searching for the wallet when they need it. The cloud server can send the prompt message to the user's mobile phone in the form of a text message to remind the user, or send the prompt message to the mobile phone software installed in the mobile phone that matches the vehicle, and send a message reminder through the mobile phone software.

[0138] In one implementation, the alarm strategy in this embodiment includes the following: sending an alarm message to the mobile terminal of the car owner; calling the user's alarm number through the intelligent customer service; automatically turning on the vehicle ventilation system. The prompt strategy includes: sending a prompt message to the mobile terminal of the car owner.

[0139] Based on Figure 1 In the specific implementation, step S102 can also be implemented in the following manner, as shown below:

[0140] Among them, the front seats may include a left front seat and a right front seat.

[0141] The left front seat space and the right front seat space are respectively subjected to a carry-over detection according to the first image.

[0142] In a specific implementation, in this embodiment, the seats in the vehicle can be divided into left seats and right seats, as shown in FIG. Figure 6 As shown, the left row of seats is the left seat, and the right row of seats is the right seat. When the vehicle is detected to be left behind, the right seat space and the left seat space are detected separately. For example, when it is detected that there are left-behind items in the left front seat space, but no left-behind items in the right front seat space, the prompt strategy is executed for the left front seat, and the posture adjustment of the left front seat is no longer performed, while the posture adjustment of the right front seat is performed, and the left-behind detection of the right rear seat space continues. In other words, the left-behind detection for the left seat and the left-behind detection for the right seat are performed separately, and they do not interfere with each other. As shown in Table 2, it is a schematic diagram of the detection results of the left seat and the right seat being left behind separately in this embodiment.

[0143] Table 2 Schematic diagram of test results

[0144]

[0145]

[0146] It can be seen from Table 2 above that when it is detected that there are objects left on any row of seats, the seat backs do not return to their initial posture, or the front seat backs of the seats with objects left do not return to their initial posture, thereby avoiding the problem of the front seat backs squeezing the objects left in the back row.

[0147] It should be noted that when a living being is detected in the left front seat space or the right front seat space, the left front seat or the right front seat will no longer be adjusted in posture to avoid squeezing the living being, and an alarm strategy will be executed to ensure the safety of the living being in the vehicle.

[0148] In one implementation, controlling the front seat to adjust the posture includes: adjusting the slide rail and backrest of the front seat to the front end, for details, refer to Figure 3 The posture of the middle front seat after adjustment.

[0149] In one embodiment, when the vehicle has three rows of seats, the schematic diagram of the posture adjustment of the seats in the vehicle is as follows: Figure 7 shown.

[0150] When the first row of seats is detected to be left behind, the first row of seats has a posture as shown in (1). When it is detected that there are no leftovers and / or life forms in the first row of seats, the first row of seats is posture adjusted, that is, the slide rails and backrests of the front row of seats are adjusted to the front end, and the adjusted seat posture is shown in (2). When it is detected that there are no leftovers and / or life forms in the second row of seats, the second row of seats is posture adjusted, and the adjusted seat posture is shown in (3). When it is detected that there are no leftovers and / or life forms in the third row of seats, the first row of seats and the second row of seats are restored to their initial postures, as shown in (1). It should be noted that the seats after posture adjustment cannot block the camera's field of view.

[0151] In one implementation, after the vehicle sends a reminder message to the owner's mobile terminal, if the owner determines that the target relic shown in the picture carried in the reminder message is not an urgently needed item, or the target relic does not affect the posture adjustment of the seat, the one-touch recovery button on the mobile terminal can be used to control the seat in the vehicle to return to its initial posture, making it convenient for the user to use the car next time.

[0152] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0153] Embodiment 2

[0154] like Figure 8 As shown, Embodiment 2 of the present invention discloses a structural schematic diagram of a vehicle left-behind detection system, the system comprising: a camera, an intelligent driving domain controller, and a central control unit, as shown below:

[0155] A camera for acquiring a first image including a front seat space of the vehicle;

[0156] An intelligent driving domain controller, configured to receive a first image transmitted from the camera, and perform a leftover detection on the front seat space based on the first image;

[0157] A central control unit, configured to control the front seats to perform posture adjustment when the intelligent driving domain controller does not detect the target remains in the first image, so that the adjusted front seats are away from the shielding area of ​​the rear seats of the vehicle;

[0158] A camera, used to obtain a second image of a rear seat space of the vehicle through the camera after the front seat posture adjustment is completed;

[0159] an intelligent driving domain controller, configured to receive the second image and perform a legacy detection on the rear seat space based on the second image;

[0160] The central control unit is used to restore the front seats to their initial posture when the intelligent driving domain controller does not detect the target remains in the second image.

[0161] In one implementation, the working principle diagram of the in-vehicle leftover detection system in this embodiment can be as follows: Fig. 9 As shown, when the in-vehicle residue detection starts, in order to obtain a clearer image, the intermediate control unit will control the lighting controller to turn on the vehicle's lights. After that, the camera will transmit the first image captured including the vehicle's front seat space to the intelligent driving domain controller. The intelligent driving domain controller will perform a residue detection on the received first image. When the intelligent driving domain controller detects that there is no target residue in the first image, it will send a seat adjustment instruction to the central control unit so that the central control unit controls the seat controller to adjust the front seats. When the intelligent driving domain controller detects that there is a target residue in the first image, the intelligent driving domain controller will send the first image to the infotainment domain controller so that the infotainment domain controller will interact with the cloud server, and then send the residue reminder information including the first image to the user terminal through the cloud server.

[0162] For the specific definition of the in-vehicle leftover detection system, please refer to the relevant definition of the in-vehicle leftover detection method mentioned above, which will not be repeated here. Each module in the above-mentioned in-vehicle leftover detection system can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0163] Embodiment 3

[0164] like Fig.10 FIG. 1 is a schematic diagram of a structure of a detection device for vehicle residues disclosed in Embodiment 3 of the present invention, as shown below:

[0165] In a specific implementation, the device in this embodiment may include the following units:

[0166] A first image acquisition unit 1001 is used to acquire a first image including a front seat space of a vehicle through a camera;

[0167] A first left-behind detection unit 1002, configured to input the first image into a recognition module to perform left-behind detection on the front seat space;

[0168] The first posture adjustment unit 1003 is used to control the front seat to adjust its posture when no target object is detected in the front seat space, so that the adjusted front seat is away from the shielding area of ​​the rear seat of the vehicle;

[0169] A second image acquisition unit 1004 is used to acquire a second image including the rear seat space of the vehicle through a camera after the front seat posture adjustment is completed;

[0170] A second left-behind detection unit 1005, for inputting the second image into the recognition module to perform left-behind detection on the rear seat space;

[0171] The second posture adjustment unit 1006 is used to restore the front seats to their initial posture when no target object is detected in the rear seat space.

[0172] In one implementation, the first legacy detection unit 1002 is specifically configured to:

[0173] Inputting the first image into a recognition module to recognize whether there is an object and / or a living being left behind in the front seat space;

[0174] If there is a living being in the front seat space, the alarm strategy will be executed;

[0175] If there is an object left in the front seat space, a prompt strategy will be implemented.

[0176] In one implementation, the second legacy detection unit 1005 is specifically configured to:

[0177] inputting the second image into a recognition module to identify whether there are objects and / or living beings left behind in the rear seat space;

[0178] If there is a living being in the rear seat space, the alarm strategy will be executed;

[0179] If there are objects left in the rear seat space, a prompt strategy will be implemented.

[0180] In one implementation, the alarm strategy includes:

[0181] Send an alarm message to the car owner's mobile terminal; or, call the user's alarm number through intelligent customer service; or, automatically open the vehicle's ventilation system;

[0182] The prompt strategy includes: sending a prompt message to the mobile terminal of the car owner.

[0183] In one implementation, the front seats include a left front seat and a right front seat;

[0184] The first legacy detection unit 1002 may also be used to:

[0185] The first image is input into the recognition module, and the left-behind seat space and the right-front seat space are respectively detected.

[0186] In one implementation, controlling the front seat to perform posture adjustment includes: adjusting the slide rail and backrest of the front seat to the frontmost position.

[0187] For the specific definition of the in-vehicle leftover detection device, please refer to the relevant definition of the in-vehicle leftover detection method mentioned above, which will not be repeated here. Each module in the above-mentioned in-vehicle leftover detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0188] Embodiment 4

[0189] Embodiment 4 of the present application discloses an electronic device, whose internal structure diagram can be shown as follows: Fig.11As shown. The electronic device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for detecting in-vehicle legacy is implemented.

[0190] In one embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0191] Acquiring a first image including a front seat space of the vehicle by means of a camera, wherein the camera is arranged near a rearview mirror position;

[0192] Inputting the first image into a recognition module to perform a leftover detection on the front seat space;

[0193] When no target remains are detected in the first image, the front seats are controlled to adjust their postures so that the adjusted front seats are away from the shielding area of ​​the rear seats of the vehicle;

[0194] After the front seat posture adjustment is completed, a second image including the rear seat space of the vehicle is acquired through the camera;

[0195] inputting the second image into a recognition module to perform a leftover detection on the rear seat space;

[0196] When no target object is detected in the second image, the front seat is restored to the initial posture.

[0197] Embodiment 5

[0198] Embodiment 5 of the present application discloses a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor in an electronic device, the electronic device can execute the steps of any embodiment of a method for detecting a vehicle left behind as disclosed in the present invention. The computer-readable storage medium can be non-volatile or volatile.

[0199] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0200] Acquiring a first image including a front seat space of the vehicle by means of a camera, wherein the camera is arranged near a rearview mirror position;

[0201] Inputting the first image into a recognition module to perform a leftover detection on the front seat space;

[0202] When no target remains are detected in the first image, the front seats are controlled to adjust their postures so that the adjusted front seats are away from the shielding area of ​​the rear seats of the vehicle;

[0203] After the front seat posture adjustment is completed, a second image including the rear seat space of the vehicle is acquired through the camera;

[0204] inputting the second image into a recognition module to perform a leftover detection on the rear seat space;

[0205] When no target object is detected in the second image, the front seat is restored to the initial posture.

[0206] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0207] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0208] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for detecting residual objects in a vehicle, characterized in that: The method comprises: Acquiring a first image including a front seat space of a vehicle; Performing a detection of objects left behind in the front seat space according to the first image, and executing vehicle control according to the detection result; and controlling the front seat to adjust its posture when no target object left behind is detected in the first image, so that the adjusted front seat is away from the shielding area of ​​the rear seat of the vehicle; After the front seat posture adjustment is completed, acquiring a second image including the rear seat space of the vehicle; A carry-over detection is performed on the rear seat space according to the second image, and vehicle control is performed according to the detection result.

2. The method for detecting vehicle residues as claimed in claim 1, characterized in that: The performing a leftover detection on the rear seat space according to the second image and executing vehicle control according to the detection result includes: When no target object is detected in the second image, the front seat is restored to an initial posture.

3. The method for detecting vehicle residues as claimed in claim 1, characterized in that: The performing a leftover detection on the front seat space according to the first image and executing vehicle control according to the detection result includes: identifying whether there are objects and / or living beings left in the front seat space according to the first image; If the life form exists in the front seat space, executing an alarm strategy; If the left object exists in the front seat space, a prompt strategy is executed.

4. The method for detecting vehicle residues as claimed in claim 1, characterized in that: The performing a leftover detection on the rear seat space according to the second image and executing vehicle control according to the detection result includes: identifying whether there are objects and / or living beings left in the rear seat space according to the second image; If the life form exists in the rear seat space, executing an alarm strategy; If the left object exists in the rear seat space, a prompt strategy is executed.

5. The method for detecting vehicle residues as claimed in claim 2, characterized in that: The vehicle further includes a third row of seats, the front row of seats being located in front of the rear row of seats, and the rear row of seats being located in front of the third row of seats; In the case where no target remains are detected in the second image, before restoring the front seat to an initial posture, the method further includes: Controlling the rear seat to adjust its posture so that the adjusted rear seat is away from the shielding area of ​​the third row seat of the vehicle; After the rear seat posture adjustment is completed, acquiring a third image including the third row seat space of the vehicle; performing a leftover detection on the third row seat space according to the third image, and executing vehicle control according to the detection result; In the case where the target object is detected in the third image, an alarm and / or prompt strategy is executed; in the case where the target object is not detected in the third image, the rear seat is restored to an initial posture.

6. The method for detecting in-vehicle residues according to any one of claims 3 to 5, characterized in that: The alarm strategy includes: Send an alarm message to the car owner's mobile terminal; or, call the user's alarm number through intelligent customer service; or, automatically open the vehicle's ventilation system; The prompt strategy includes: sending a prompt message to the mobile terminal of the car owner.

7. The method for detecting vehicle residues as claimed in claim 1, characterized in that: The front seats include a left front seat and a right front seat; The performing of a leftover detection on the front seat space according to the first image includes: The left front seat space and the right front seat space are respectively subjected to a carry-over detection according to the first image.

8. The method for detecting vehicle residues as claimed in claim 1, characterized in that: The controlling the front seat to adjust the posture includes: adjusting the slide rail and the backrest of the front seat to the frontmost position.

9. A vehicle left-behind detection system, characterized in that: The system includes: a camera, a smart driving domain controller, a central control unit, The camera is used to obtain a first image including the front seat space of the vehicle; The intelligent driving domain controller is used to receive a first image transmitted by a camera, and perform a legacy detection on the front seat space based on the first image; The central control unit is configured to control the front seats to perform posture adjustment when the intelligent driving domain controller does not detect the target remains in the first image, so that the adjusted front seats are away from the shielding area of ​​the rear seats of the vehicle; The camera is used to obtain a second image including the rear seat space of the vehicle through the camera after the front seat posture adjustment is completed; The intelligent driving domain controller is used to receive the second image and perform legacy detection on the rear seat space based on the second image; The central control unit is used to restore the front seats to their initial posture when the intelligent driving domain controller does not detect the target remains in the second image.

10. A detection device for leftovers in a vehicle, characterized in that: include: A first image acquisition unit, configured to acquire a first image including a front seat space of the vehicle through a camera; a first left-behind detection unit, configured to input the first image into a recognition module to perform left-behind detection on the front seat space; a first posture adjustment unit, configured to control the front seat to perform posture adjustment when no target object is detected in the front seat space, so that the adjusted front seat is away from a shielding area of ​​a rear seat of the vehicle; a second image acquisition unit, configured to acquire, through the camera, a second image including a rear seat space of the vehicle after the front seat posture adjustment is completed; a second left-behind detection unit, configured to input the second image into a recognition module to perform left-behind detection on the rear seat space; The second posture adjustment unit is used to restore the front seat to an initial posture when no target object is detected in the rear seat space.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for detecting in-vehicle residues as claimed in any one of claims 1 to 8 is implemented.

12. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting in-vehicle residues as claimed in any one of claims 1 to 8 is implemented.

Images