In-vehicle article taking method and system combining seat adjustment and visual identification

Abstract

The invention provides an in-vehicle article taking method and system combining seat adjustment and visual identification, and belongs to the technical field of vehicle control. The method comprises the following steps: periodically acquiring rear row area images through an in-vehicle camera, identifying article types and position frames in the images by using a target detection algorithm, and outputting article confidence and article position information; a preset datum point of the front row of seats is used as a reference, and an accessible area range is calculated according to the front and back positions, the backrest angle and the height of the seats; calculating the spatial relationship between the article position information and the accessible area range in real time, and if the target article exceeds the accessible area range, determining that the target article cannot be directly taken and used; and when the target article cannot be directly taken and used, calculating and obtaining a seat adjusting action so that the target article can enter an accessible range of the front-row passengers after the seat adjusting action is completed, and generating a control command sequence according to the seat adjusting action and sending the control command sequence to a seat control unit for execution. The intelligent auxiliary object taking device has the intelligent auxiliary object taking function.

Description

Technical Field

[0001] This invention relates to the field of vehicle control technology, and in particular to a method and system for retrieving items inside a vehicle by combining seat adjustment and visual recognition. Background Technology

[0002] In real-world driving scenarios, drivers or front-seat passengers often need to retrieve items from the rear seats, such as bags, drinks, documents, or umbrellas. However, in traditional designs, this often requires passengers to unbuckle their seatbelts, turn around, or even get out of the car, which affects both driving safety and inconvenience.

[0003] Existing technologies often predict whether front-seat occupants intend to move backward by detecting their body posture, and then assist by moving the seat when a tendency to retrieve an object is detected. However, these methods rely solely on inferences about occupant intentions and cannot directly identify the presence or location of objects, nor can they provide targeted, automated assistance for retrieving items. Summary of the Invention

[0004] The present invention aims to solve at least one of the above-mentioned problems in the prior art by proposing a new method that combines visual recognition and seat adjustment, enabling the vehicle to actively sense the position of items in the rear seats and automatically plan seat adjustment actions under the condition of meeting safety constraints, thereby helping passengers to retrieve items conveniently and safely.

[0005] In a first aspect, embodiments of the present invention provide a method for retrieving items inside a vehicle that combines seat adjustment and visual recognition, including:

[0006] The system periodically collects images of the rear seat area using an in-vehicle camera, uses an object detection algorithm to identify the type and location bounding box of objects in the images, and outputs the confidence score and location information of the objects.

[0007] Using the preset reference point of the front seats as a reference, the reachable area is calculated based on the seat's fore-and-aft position, backrest angle, and height.

[0008] The system calculates the spatial relationship between the location information of an item and the reachable area in real time. If the target item is outside the reachable area, it is confirmed that the target item cannot be directly retrieved.

[0009] When the target item is not directly accessible, the seat adjustment action is calculated and obtained so that the target item can enter the reach range of the front passenger after the seat adjustment action is completed. A control command sequence is generated based on the seat adjustment action and sent to the seat control unit for execution.

[0010] In a preferred embodiment, the steps of periodically acquiring images of the rear seat area via an in-vehicle camera, identifying the type and location bounding box of objects in the images using a target detection algorithm, and outputting object confidence and object location information include:

[0011] By combining the calibration parameters of the in-vehicle camera with a monocular depth estimation algorithm or a binocular matching algorithm, the spatial position of the object relative to the vehicle is estimated, and the position information of the object in the vehicle coordinate system is obtained.

[0012] In a preferred embodiment, in the step of calculating the reachable area range based on the front seat's preset reference point, the seat's fore-and-aft position, backrest angle, and height:

[0013] The accessible area is a three-dimensional spatial region centered on a preset reference point of the front seats, including: the maximum reachable distance in the front-to-back direction, the maximum reachable distance in the up-down direction, and the maximum reachable distance in the left-to-right direction.

[0014] In a preferred embodiment, the step of calculating and obtaining a seat adjustment action so that the target item can enter the reachable range of the front passenger after the seat adjustment action is completed, and generating a control command sequence based on the seat adjustment action and sending it to the seat control unit for execution, includes:

[0015] When the target item is not directly accessible, read the vehicle seat's adjustment capabilities, including: maximum forward movement, maximum backrest tilt angle, and maximum height adjustment.

[0016] Within the adjustment range of the vehicle seat, generate several action combinations, predict the new position of the front seat reference point after each action combination is adjusted, calculate the new accessible area corresponding to the new position, and determine whether the target item has entered the new accessible area.

[0017] When a target action combination enables a target item to enter a new accessible area, the target action combination is confirmed to enable the target item to enter the accessible range of the front passenger after the target action combination is completed. A control command sequence is generated based on the target action combination and sent to the seat control unit for execution.

[0018] In a preferred embodiment, the step of confirming that the target action combination enables the target item to enter a new accessible area after the target action combination is completed, and generating a control command sequence based on the target action combination and sending it to the seat control unit for execution, is as follows:

[0019] When multiple target action combinations can bring the target item into the corresponding new accessible area, the target action combination with the smallest total adjustment amount, the least impact on occupant comfort, or the smooth adjustment process and short execution time is selected as the optimal seat adjustment action. A control command sequence is generated based on the optimal seat adjustment action and sent to the seat control unit for execution.

[0020] In a preferred embodiment, when the target item is not directly accessible, the following steps are performed: calculating and obtaining a seat adjustment action so that the target item can enter the reachable range of the front passenger after the seat adjustment action is completed; and generating a control command sequence based on the seat adjustment action and sending it to the seat control unit for execution.

[0021] If any preset safety condition is not met, the step of generating a control command sequence based on the seat adjustment action and sending it to the seat control unit will not be executed, and a prompt will be issued indicating that the safety condition is not met.

[0022] In a preferred embodiment, it further includes:

[0023] Once the seat adjustment is complete, the in-vehicle camera detects the location of the target item and calculates the current seat access area. If the target item has entered the current seat access area, a prompt is issued that the item can be retrieved; if the target item has not entered the current seat access area, a prompt is issued to manually retrieve the item or re-trigger the planning.

[0024] In a second aspect, embodiments of the present invention provide an in-vehicle item retrieval system combining seat adjustment and visual recognition, configured to implement any of the methods described in the first aspect, the system comprising:

[0025] The object recognition and positioning module is used to periodically collect images of the rear seat area through the in-vehicle camera, use the target detection algorithm to identify the type and location box of the object in the image, and output the object confidence score and the object location information.

[0026] The Accessible Zone Calculation Module is used to calculate the accessible zone range based on the preset reference point of the front seats, the fore-and-aft position of the seats, the backrest angle, and the height.

[0027] The accessibility analysis module is used to calculate the spatial relationship between the location information of an item and the accessible area in real time. If the target item is outside the accessible area, it is confirmed that the target item cannot be directly taken.

[0028] The seat motion planning module is used to calculate and obtain seat adjustment actions when the target item cannot be directly accessed, so that the target item can enter the reachable range of the front passenger after the seat adjustment action is completed, and generate a control command sequence based on the seat adjustment action and send it to the seat control unit for execution.

[0029] Thirdly, embodiments of the present invention provide an electronic device, including:

[0030] One or more processors;

[0031] Memory, used to store one or more programs;

[0032] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the methods described in the first aspect.

[0033] Fourthly, embodiments of the present invention provide a computer-readable medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described in the first aspect.

[0034] Beneficial effects of this invention:

[0035] This invention identifies the location of items in the rear seats and calculates spatial relationships using an in-vehicle camera without adding any mechanical devices. It dynamically establishes an "accessible zone" based on the position of the occupants' seats to determine whether the items are reachable, and plans the optimal seat action sequence that meets the vehicle's safety constraints. Thus, it has the function of intelligently assisting in retrieving items. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the overall process of a method for retrieving items inside a vehicle that combines seat adjustment and visual recognition, provided by an embodiment of the present invention.

[0037] Figure 2 This is a schematic flowchart illustrating a method for retrieving in-vehicle items that combines seat adjustment and visual recognition, as provided in an embodiment of the present invention.

[0038] Figure 3 This is a schematic flowchart of a specific implementation of step S4 provided in an embodiment of the present invention.

[0039] Figure 4 This is a schematic diagram of an in-vehicle item retrieval system that combines seat adjustment and visual recognition, provided as an embodiment of the present invention.

[0040] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0041] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0042] Where there is no conflict, the various embodiments of the present invention and the features thereof may be combined with each other.

[0043] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0044] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0045] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein.

[0046] In the technical solution of this invention, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information all comply with relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution follows relevant national laws and regulations (e.g., the "Information Security Technology - Personal Information Security Specification"). For example: appropriate measures are taken for personal information access control; restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; and explicit identity targeting is eliminated when using personal information to avoid precisely locating a specific individual.

[0047] Some abbreviations and key terms in this invention are defined as follows:

[0048] Visual recognition module: A system that uses in-vehicle cameras to capture images and detect objects in the rear passenger area.

[0049] Accessible area: refers to the space that front-seat passengers can reach with their arms outstretched naturally without turning around while in a normal sitting position.

[0050] Seat adjustment parameters: These refer to the electrically adjustable parameters such as the seat's fore-and-aft position, backrest angle, and height.

[0051] Motion planning module: This module calculates the relationship between the relative position of objects and the seat adjustment action, and is used to generate the optimal motion sequence.

[0052] Safety constraints: These include conditions such as vehicle speed, gear position, and the status of rear passengers, used to ensure the safety of the operation.

[0053] This invention proposes a method and system for retrieving items from inside a vehicle by combining seat adjustment and visual recognition. The method and system utilize existing in-vehicle cameras, electric seat adjustment mechanisms, and onboard control units to achieve a complete closed loop of "perception-judgment-execution-feedback." The overall process includes four core stages: item identification and location, accessibility analysis, seat motion planning, and safety judgment and execution feedback.

[0054] Figure 1 This is a schematic diagram illustrating the overall process of a method for retrieving items inside a vehicle that combines seat adjustment and visual recognition, as provided in an embodiment of the present invention. Figure 1 As shown, the overall process of this method includes:

[0055] The process begins;

[0056] Capture images inside the vehicle;

[0057] Identify rear-row items and calculate their spatial location;

[0058] Determine if "Is the item within reach?" If yes, prompt the passenger that they can retrieve the item directly; otherwise, calculate the seat adjustment action.

[0059] Determine if "Safety conditions are met?" If not, terminate the process and display a safety constraint message; otherwise, perform seat adjustment.

[0060] Re-identify the item's location;

[0061] Determine if the item has entered the accessible area. If yes, display "Item is available" and the process ends; otherwise, prompt the user to replan or manually retrieve the item and the process ends.

[0062] In the following embodiments of the present invention, for ease of description, the vehicle control unit is used as the executing entity. The executing entity may also be a software module, or other electronic devices capable of performing the following functions.

[0063] Figure 2 This is a flowchart illustrating a method for retrieving in-vehicle items that combines seat adjustment and visual recognition, as provided in an embodiment of the present invention. Figure 2 As shown, the method includes:

[0064] Step S1: Periodically collect images of the rear seat area using the in-vehicle camera, use the object detection algorithm to identify the type and location bounding box of the items in the image, and output the item confidence score and item location information;

[0065] Step S2: Using the preset reference point of the front seat as a reference, calculate the reachable area range based on the seat's fore-and-aft position, backrest angle, and height.

[0066] Step S3: Calculate the spatial relationship between the item's location information and the accessible area in real time. If the target item is outside the accessible area, confirm that the target item cannot be directly taken.

[0067] Step S4: When the target item cannot be directly accessed, calculate and obtain the seat adjustment action so that the target item can enter the reachable range of the front passenger after the seat adjustment action is completed, and generate a control command sequence based on the seat adjustment action and send it to the seat control unit for execution.

[0068] This invention automatically identifies items and their location information in the rear row area based on the acquired rear row area image using the above steps and a target detection algorithm. It calculates the reachable area range based on the front seat information. When it is determined that an item is outside the reachable area range based on the item's location information, it calculates a seat adjustment action command and sends it to the seat control unit for execution, so that the item enters the occupant's reachable range after the seat adjustment is completed. Therefore, this invention has the function of intelligently assisting in retrieving items.

[0069] The in-vehicle cameras include those positioned on the roof or seatbacks; the object detection algorithm can be based on a convolutional neural network model; the preset reference point S0 for the front seats is, for example, the midpoint of the seat track or the geometric center of the seat cushion; if an item has fallen into the accessible area, the occupant can be prompted that "the item can be taken directly"; if the item is outside the accessible area, the seat motion planning stage begins.

[0070] In some embodiments, step S1 involves periodically acquiring images of the rear seat area using an in-vehicle camera, identifying the type and location bounding box of objects in the images using an object detection algorithm, and outputting the object confidence score and object location information.

[0071] By combining the calibration parameters of the in-vehicle camera (installation position, field of view, focal length, etc.) with a monocular depth estimation algorithm or a binocular matching algorithm, the spatial position of the object relative to the vehicle is estimated, and the position information P(x, y, z) of the object in the vehicle coordinate system is obtained.

[0072] Where: x represents the left and right offset, y represents the height position, and z represents the front and back distance (i.e., the longitudinal distance between the item and the front passenger).

[0073] In some embodiments, step S1, which involves periodically acquiring images of the rear seat area using an in-vehicle camera, identifying the type and location bounding box of objects in the images using an object detection algorithm, and outputting the object confidence score and object location information, further includes:

[0074] The system filters items based on their confidence level, outputting the confidence levels and corresponding location information of items whose confidence levels exceed a preset confidence threshold. By filtering the recognition results based on confidence, false positives and occluded targets are eliminated, retaining only reliable item coordinates for subsequent analysis.

[0075] In some embodiments, in step S2, the accessible area is calculated based on the seat's fore-and-aft position, backrest angle, and height, with reference to a preset reference point of the front seat.

[0076] The accessible area is defined by a reference point preset for the front seats. The three-dimensional spatial region centered on the object includes: the maximum reachable distance Lx in the front-back direction (approximately 400–600 mm), the maximum reachable distance Ly in the up-down direction (approximately 200 mm), and the maximum reachable distance Lz in the left-right direction (approximately 150 mm).

[0077] These values ​​can be automatically adjusted according to the occupant's body shape and seat posture. As the seat moves forward and the backrest tilts forward, the accessible area moves away from the rear seats; when the seat moves backward or the backrest tilts back, the accessible area expands towards the rear seats.

[0078] In some embodiments, such as Figure 3 As shown, step S4, when the target item is not directly accessible, calculates and obtains a seat adjustment action so that the target item can enter the reachable range of the front passenger after the seat adjustment action is completed, and generates a control command sequence based on the seat adjustment action and sends it to the seat control unit for execution. The steps include:

[0079] Step S41: When the target item cannot be directly accessed, read the adjustment capabilities of the vehicle seat, including: maximum forward movement (Xmax), maximum backrest tilt angle (Θmax), and maximum lifting height (Hmax).

[0080] Step S42: Generate several action combinations within the adjustment range of the vehicle seat, and predict the front seat reference point after adjustment for each action combination. The new location is determined and the corresponding new accessible area is calculated to determine whether the target item has entered the new accessible area.

[0081] Step S43: When the target action combination enables the target item to enter the new accessible area, confirm that the target action combination enables the target item to enter the accessible range of the front passenger after the target action combination is completed, and generate a control command sequence according to the target action combination and send it to the seat control unit for execution.

[0082] The adjustability of the vehicle seat constitutes the seat's own safety boundary, and all planned schemes must not exceed these safety boundaries. Within the permissible safety boundary range, several action combinations are implemented, such as: moving forward 50 mm + backrest tilting forward 5°; moving forward 70 mm; moving forward 40 mm + raising 15 mm, etc. The final scheme is converted into a specific control command sequence, such as "move forward 60 mm, backrest tilting forward 3°", and sent to the seat control unit for execution.

[0083] In some embodiments, in step S43, when the target action combination enables the target item to enter a new accessible area, it is confirmed that the target action combination enables the target item to enter the accessible range of the front passenger after the target action combination is completed, and a control command sequence is generated based on the target action combination and sent to the seat control unit for execution.

[0084] When multiple target action combinations can bring the target item into the corresponding new accessible area, the target action combination with the smallest total adjustment amount, the least impact on occupant comfort, or the smooth adjustment process and short execution time is selected as the optimal seat adjustment action. A control command sequence is generated based on the optimal seat adjustment action and sent to the seat control unit for execution.

[0085] In some embodiments, in step S4, when the target item is not directly accessible, a seat adjustment action is calculated and obtained so that the target item can enter the reachable range of the front passenger after the seat adjustment action is completed. A control command sequence is generated based on the seat adjustment action and sent to the seat control unit for execution.

[0086] If any preset safety condition is not met, the step of generating a control command sequence based on the seat adjustment action and sending it to the seat control unit will not be executed, and a prompt indicating that the safety condition is not met will be issued. The preset safety conditions include: vehicle speed below 5 km / h, gear in P or low D, no occupant or child seat detected in the rear, and seat movement path will not interfere with other vehicle interior structures.

[0087] In some embodiments, the method further includes: step S5, after the seat adjustment action is completed, using the in-vehicle camera to detect the position of the target item, calculating the current seat access area range, and if the target item has entered the current seat access area range, issuing a prompt that the item can be retrieved; if the target item has not entered the current seat access area range, issuing a prompt to manually retrieve the item or re-trigger the planning.

[0088] Based on the same inventive concept, such as Figure 4 As shown, this embodiment of the invention also provides an in-vehicle item retrieval system combining seat adjustment and visual recognition, configured to implement any of the methods described in the above embodiments, the system comprising:

[0089] The object recognition and positioning module is used to periodically collect images of the rear seat area through the in-vehicle camera, use the target detection algorithm to identify the type and location box of the object in the image, and output the object confidence score and the object location information.

[0090] The Accessible Zone Calculation Module is used to calculate the accessible zone range based on the preset reference point of the front seats, the fore-and-aft position of the seats, the backrest angle, and the height.

[0091] The accessibility analysis module is used to calculate the spatial relationship between the location information of an item and the accessible area in real time. If the target item is outside the accessible area, it is confirmed that the target item cannot be directly taken.

[0092] The seat motion planning module is used to calculate and obtain seat adjustment actions when the target item cannot be directly accessed, so that the target item can enter the reachable range of the front passenger after the seat adjustment action is completed, and generate a control command sequence based on the seat adjustment action and send it to the seat control unit for execution.

[0093] Based on the same inventive concept, embodiments of the present invention also provide an electronic device. Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Figure 5 As shown, an embodiment of the present invention provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement any of the methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0094] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0095] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0096] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0097] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable medium. This computer-readable medium stores a computer program, wherein, when executed by a processor, the program implements the steps of any of the methods described in the above embodiments. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0098] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0099] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0100] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0101] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0102] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0103] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0104] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0105] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0106] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0107] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A method for retrieving in-vehicle items combining seat adjustment and visual recognition, characterized in that, include: The system periodically collects images of the rear seat area using an in-vehicle camera, uses an object detection algorithm to identify the type and location bounding box of objects in the images, and outputs the confidence score and location information of the objects. Using the preset reference point of the front seats as a reference, the reachable area is calculated based on the seat's fore-and-aft position, backrest angle, and height. The system calculates the spatial relationship between the location information of an item and the reachable area in real time. If the target item is outside the reachable area, it is confirmed that the target item cannot be directly retrieved. When the target item is not directly accessible, the seat adjustment action is calculated and obtained so that the target item can enter the reach range of the front passenger after the seat adjustment action is completed. A control command sequence is generated based on the seat adjustment action and sent to the seat control unit for execution.

2. The method according to claim 1, characterized in that, In the step of periodically acquiring images of the rear seat area via an in-vehicle camera, using a target detection algorithm to identify the type and location bounding box of objects in the images, and outputting the object confidence score and object location information: By combining the calibration parameters of the in-vehicle camera with a monocular depth estimation algorithm or a binocular matching algorithm, the spatial position of the object relative to the vehicle is estimated, and the position information of the object in the vehicle coordinate system is obtained.

3. The method according to claim 1, characterized in that, The step of calculating the reachable area range based on the preset reference point of the front seats, the fore-and-aft position of the seats, the backrest angle, and the height is as follows: The accessible area is a three-dimensional spatial region centered on a preset reference point of the front seats, including: the maximum reachable distance in the front-to-back direction, the maximum reachable distance in the up-down direction, and the maximum reachable distance in the left-to-right direction.

4. The method according to claim 1, characterized in that, When the target item is not directly accessible, the steps of calculating and obtaining a seat adjustment action so that the target item can enter the reachable range of the front passenger after the seat adjustment action is completed, and generating a control command sequence based on the seat adjustment action and sending it to the seat control unit for execution include: When the target item is not directly accessible, read the vehicle seat's adjustment capabilities, including: maximum forward movement, maximum backrest tilt angle, and maximum height adjustment. Within the adjustment range of the vehicle seat, generate several action combinations, predict the new position of the front seat reference point after each action combination is adjusted, calculate the new accessible area corresponding to the new position, and determine whether the target item has entered the new accessible area. When a target action combination enables a target item to enter a new accessible area, the target action combination is confirmed to enable the target item to enter the accessible range of the front passenger after the target action combination is completed. A control command sequence is generated based on the target action combination and sent to the seat control unit for execution.

5. The method according to claim 4, characterized in that, When the target action combination enables the target item to enter a new accessible area, confirming that the target action combination allows the target item to enter the accessible range of the front passenger after the target action combination is completed, and generating a control command sequence based on the target action combination and sending it to the seat control unit for execution: When multiple target action combinations can bring the target item into the corresponding new accessible area, the target action combination with the smallest total adjustment amount, the least impact on occupant comfort, or the smooth adjustment process and short execution time is selected as the optimal seat adjustment action. A control command sequence is generated based on the optimal seat adjustment action and sent to the seat control unit for execution.

6. The method according to claim 4, characterized in that, When the target item is not directly accessible, the following steps are performed: First, calculate and obtain the seat adjustment action so that the target item can be brought within reach of the front passenger after the seat adjustment action is completed. Then, generate a control command sequence based on the seat adjustment action and send it to the seat control unit for execution. If any preset safety condition is not met, the step of generating a control command sequence based on the seat adjustment action and sending it to the seat control unit will not be executed, and a prompt will be issued indicating that the safety condition is not met.

7. The method according to claim 1, characterized in that, Also includes: Once the seat adjustment is complete, the in-vehicle camera detects the location of the target item and calculates the current seat access area. If the target item has entered the current seat access area, a prompt is issued that the item can be retrieved; if the target item has not entered the current seat access area, a prompt is issued to manually retrieve the item or re-trigger the planning.

8. A vehicle in-vehicle item retrieval system combining seat adjustment and visual recognition, characterized in that, The system, configured to implement the method as described in any one of claims 1 to 7, comprises: The object recognition and positioning module is used to periodically collect images of the rear seat area through the in-vehicle camera, use the target detection algorithm to identify the type and location box of the object in the image, and output the object confidence score and the object location information. The Accessible Zone Calculation Module is used to calculate the accessible zone range based on the preset reference point of the front seats, the fore-and-aft position of the seats, the backrest angle, and the height. The accessibility analysis module is used to calculate the spatial relationship between the location information of an item and the accessible area in real time. If the target item is outside the accessible area, it is confirmed that the target item cannot be directly taken. The seat motion planning module is used to calculate and obtain seat adjustment actions when the target item cannot be directly accessed, so that the target item can enter the reachable range of the front passenger after the seat adjustment action is completed, and generate a control command sequence based on the seat adjustment action and send it to the seat control unit for execution.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images