Time-division multiplexing method, device, electronic device and storage medium for perception system

Abstract

The disclosed embodiment discloses a time-division multiplexing method, device, electronic device and storage medium of a perception system, wherein the method comprises: in response to receiving a trigger instruction to enter a parking mode, controlling a video channel to switch from a cockpit channel to a parking channel; acquiring parking image data based on the parking channel; and determining a parking perception result based on the parking image data. The disclosed embodiment implements time-division multiplexing of the perception system by switching the video channel between the cockpit channel and the parking channel, so that the cockpit perception and the parking perception can share a domain controller, effectively improving resource utilization, saving domain controllers and reducing costs.

Description

Technical Field

[0001] The present disclosure relates to computer vision technology, and in particular to a time-division multiplexing method, device, electronic device and storage medium for a perception system. Background Art

[0002] With the continuous development of computer vision technology, cockpit perception systems for cockpits and parking perception systems for automatic parking are installed in autonomous driving, semi-autonomous driving and other vehicles to meet the perception needs of smart cockpits and automatic parking respectively. The cockpit perception system is implemented based on the cockpit domain controller, and the parking perception system is implemented based on the parking domain controller. Two independent domain controllers are required, resulting in low domain controller resource utilization and high cost. Summary of the invention

[0003] In order to solve the above technical problems of low resource utilization, high cost, etc., the present disclosure is proposed. The embodiments of the present disclosure provide a time-division multiplexing method, device, electronic device and storage medium for a perception system.

[0004] According to one aspect of an embodiment of the present disclosure, a time-division multiplexing method of a perception system is provided, comprising: in response to receiving a trigger instruction to enter a parking mode, controlling a video channel to switch from a cockpit channel to a parking channel; acquiring parking image data based on the parking channel; and determining a parking perception result based on the parking image data.

[0005] According to another aspect of an embodiment of the present disclosure, a time-sharing multiplexing device of a perception system is provided, comprising: a first control module, for controlling a video channel to switch from a cockpit channel to a parking channel in response to receiving a trigger instruction to enter a parking mode; a first acquisition module, for acquiring parking image data based on the parking channel; and a first processing module, for determining a parking perception result based on the parking image data.

[0006] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the time-division multiplexing method of the perception system described in any of the above embodiments of the present disclosure.

[0007] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the time-division multiplexing method of the perception system described in any of the above embodiments of the present disclosure.

[0008] Based on the time-sharing multiplexing method, device, electronic device and storage medium of the perception system provided by the above embodiments of the present disclosure, the time-sharing multiplexing of the perception system is realized by switching the video channel between the cockpit channel and the parking channel, so that the cockpit perception and parking perception can share a domain controller, effectively improving resource utilization, saving domain controllers and reducing costs.

[0009] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is an exemplary application scenario of the time-division multiplexing method of the perception system provided by the present disclosure;

[0012] Figure 2 is a flowchart of a time-division multiplexing method of a perception system provided by an exemplary embodiment of the present disclosure;

[0013] Figure 3 is a flow chart of a time-division multiplexing method of a perception system provided by another exemplary embodiment of the present disclosure;

[0014] Figure 4 is a flowchart of a time-division multiplexing method of a perception system provided by yet another exemplary embodiment of the present disclosure;

[0015] Figure 5 is a flowchart of a time-division multiplexing method of a perception system provided by yet another exemplary embodiment of the present disclosure;

[0016] Figure 6 is a flowchart of a time-division multiplexing method of a perception system provided by another exemplary embodiment of the present disclosure;

[0017] Figure 7 is a structural schematic diagram of a time-division multiplexing device of a perception system provided by an exemplary embodiment of the present disclosure;

[0018] Figure 8 is a structural schematic diagram of a time-division multiplexing device of a perception system provided by another exemplary embodiment of the present disclosure;

[0019] Fig. 9 is a structural diagram of a first control module 501 provided by an exemplary embodiment of the present disclosure;

[0020] Fig.10 is a structural schematic diagram of a time-division multiplexing device of a perception system provided by yet another exemplary embodiment of the present disclosure;

[0021] Fig.11 is a structural schematic diagram of a time-division multiplexing device of a perception system provided by another exemplary embodiment of the present disclosure;

[0022] Fig.12 It is a structural schematic diagram of an application embodiment of the electronic device disclosed in the present invention. DETAILED DESCRIPTION

[0023] Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0024] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0025] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0026] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0027] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0028] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0029] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0030] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0031] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0032] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0033] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0034] The embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0035] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0036] SUMMARY OF THE DISCLOSURE

[0037] In the process of implementing the present disclosure, the inventors discovered that a cockpit perception system for the cockpit and a parking perception system for automatic parking are provided on vehicles such as autonomous driving and semi-autonomous driving, which are used to meet the perception needs of the smart cockpit and automatic parking, respectively. The cockpit perception system is implemented based on a cockpit domain controller, and the parking perception system is implemented based on a parking domain controller, requiring two independent domain controllers, resulting in low domain controller resource utilization and high cost.

[0038] Exemplary Overview

[0039] Figure 1 This is an exemplary application scenario of the time-division multiplexing method of the perception system provided in the present disclosure.

[0040] In this scenario, the vehicle is provided with in-cabin visual sensors (cameras 1-4) and out-cabin visual sensors (cameras 5-8). The in-cabin and out-cabin are each a group, and are respectively connected to the time-sharing multiplexing device of the perception system for executing the time-sharing multiplexing method of the perception system disclosed in the present invention through a deserializer. The device controls the video channel to switch to different groups in different situations according to the time-sharing multiplexing rules to receive the image data of the visual sensor of the group. The image data may include a single frame or multiple frames of images, and may also include a video, which may be set according to actual needs. The deserializer is an interface circuit in high-speed data communication. When used in conjunction with a serializer, a single-channel or dual-channel stream can be converted into parallel data. The specific principle will not be repeated. Specifically, when a trigger instruction to enter the parking mode is received, the video channel can be controlled to switch to the parking channel. In this scenario, the parking channel refers to the video channel corresponding to the camera 5-8 of the out-cabin group, and then the parking image data can be obtained based on the parking channel, and the parking perception result can be determined based on the parking image data, which is used to assist parking, parking perception such as parking space detection, obstacle recognition, and so on. When the parking mode ends, the video channel can be controlled to switch to the cockpit channel to receive the image data of the visual sensor in the cabin and perform perception in the cabin, such as face recognition, age and gender recognition, phone call event recognition, smoking event recognition, fatigue detection, emotion recognition, sleep detection, gesture recognition, etc. The specific perception content can be set according to actual needs. Among them, the parking mode can include manual parking mode and automatic parking mode, which are not specifically limited. The trigger instruction for entering the parking mode can include any possible trigger instruction, such as a button trigger instruction, a voice trigger instruction, etc. The button trigger instruction is a trigger instruction generated by the user manually triggering a hardware button (such as a parking button set on the vehicle) or a virtual button (such as a button on the screen, specifically such as a selection box on the interface in the central control screen), and the voice trigger instruction is a trigger instruction generated by the user through voice interaction. In actual applications, the visual sensors inside and outside the cabin can be set according to actual needs, not limited to the above-mentioned cameras 1-8. By utilizing the time-sharing multiplexing method of the perception system disclosed in the present invention, time-sharing multiplexing of the perception system can be achieved, so that cabin perception and parking perception can share a domain controller, effectively improving resource utilization, saving domain controllers and reducing costs.

[0041] In actual applications, visual sensors can be divided into more groups according to actual needs. For example, they can also include an outside unlocking camera, which can be used as a group corresponding to an outside unlocking channel, or can be used as a group with an in-cabin camera. Before the user gets on the vehicle, the in-cabin camera is in a closed state, and the cabin channel only obtains image data from the outside unlocking camera for unlocking perception, and controls the vehicle to unlock so that the user can enter the vehicle. When the vehicle is started, the unlocking camera can be controlled to be turned off and the in-cabin camera can be turned on, and the cabin channel obtains the image data of the in-cabin camera for cabin perception. The specific settings can be based on actual needs and are not limited in this disclosure.

[0042] Exemplary Methods

[0043] Figure 2 FIG. 1 is a flow chart of a time-division multiplexing method of a perception system provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices, such as a vehicle-mounted computing platform, such as Figure 2 As shown, the following steps are included:

[0044] Step 201 , in response to receiving a trigger instruction to enter a parking mode, controlling a video channel to switch from a cockpit channel to a parking channel.

[0045] Among them, the parking mode can be a manual parking mode or an automatic parking mode, which is not specifically limited. The trigger instruction for entering the parking mode may include any possible trigger instruction, such as a button trigger instruction, a voice trigger instruction, and the like. The button trigger instruction is a trigger instruction generated by the user manually triggering a hardware button (such as a parking button set on the vehicle) or a virtual button (such as a button on the screen, specifically such as a selection box on the interface in the central control screen), and the voice trigger instruction is a trigger instruction generated by the user through voice interaction. The video channel refers to a connection channel connected to a visual sensor (such as a camera) to obtain image data from the visual sensor, which may include a cockpit channel and a parking channel. In actual applications, the video channel may also include other types of video channels, which are not specifically limited. The cockpit channel refers to a video channel connected to a visual sensor in the cockpit, and the parking channel refers to a video channel connected to a visual sensor outside the vehicle. The specific connection method may be any feasible method, which is not limited in the present disclosure. The switching of video channels can be achieved through a time-sharing switch, which can be implemented through hardware or software. For example, the status (such as enabled status and standby status) can be set for the cockpit channel and the parking channel, and the switching of video channels can be achieved through the configuration of the status.

[0046] When the vehicle is in non-parking mode (such as normal driving), the video channel can be in the cockpit channel, which is used to receive image data from the visual sensor in the cabin and perform perception in the cabin, such as face recognition, age and gender recognition, phone call event recognition, smoking event recognition, fatigue detection, emotion recognition, sleep detection, gesture recognition, etc. The specific perception content can be set according to actual needs. When a trigger instruction to enter the parking mode is received, indicating that the parking mode needs to be entered, the video channel can be controlled to switch from the cockpit channel to the parking channel for perception of the environment outside the vehicle.

[0047] Step 202: Acquire parking image data based on the parking channel.

[0048] The parking image data may include image data or video data collected by the external visual sensor. After entering the parking mode, the external visual sensor needs to collect the image of the external environment to assist parking. For example, the external visual sensor is set as Figure 1 The four cameras shown in the figure, namely, front, rear, left and right, can obtain surround view image data of the vehicle from the four cameras based on the parking channel as parking image data. The specific amount of parking image data can be set according to actual needs and is not limited to the image data collected by the four cameras.

[0049] Step 203: Determine a parking perception result based on the parking image data.

[0050] After the parking image data is acquired, the parking perception result can be obtained by performing perception processing based on a preset perception algorithm or model. The specific perception algorithm or model can adopt any feasible algorithm or model, which is not limited in the present disclosure. For example, the parking space detection, obstacle recognition and other perception processing can be performed based on the parking image data.

[0051] The time-sharing multiplexing method of the perception system provided in this embodiment realizes the time-sharing multiplexing of the perception system by switching the video channel between the cockpit channel and the parking channel, so that the cockpit perception and the parking perception can share a domain controller, effectively improving resource utilization, saving domain controllers and reducing costs.

[0052] In an alternative example, Figure 3 FIG. 2 is a flow chart of a time-division multiplexing method of a perception system provided by another exemplary embodiment of the present disclosure. In this example, step 201 may specifically include the following steps:

[0053] Step 2011, in response to receiving a trigger instruction to enter the parking mode, the state of the cabin channel is switched from the enabled state to the standby state, and the state of the parking channel is switched from the standby state to the enabled state.

[0054] Among them, the cockpit channel and the parking channel are set to indicate whether the video channel is available, so as to realize the switching between the cockpit channel and the parking channel. The standby state indicates that the video channel is not currently enabled, and the enabled state indicates that the video channel is currently available. The specific representation of the standby state and the enabled state can be set according to actual needs. After receiving the trigger instruction to enter the parking mode, the cockpit channel and the parking channel are switched by switching the state of the cockpit channel from the enabled state to the standby state and switching the state of the parking channel from the standby state to the enabled state.

[0055] Figure 4 It is a flowchart of a time-division multiplexing method of a perception system provided by yet another exemplary embodiment of the present disclosure.

[0056] In an optional example, the method of the present disclosure further includes:

[0057] Step 204 , in response to the parking mode being terminated, the video channel is controlled to switch from the parking channel to the cockpit channel.

[0058] Specifically, the parking mode end trigger can be any feasible way, such as detecting that the vehicle has parked in the parking space, detecting the user's stop parking button command or voice command, etc., which can be set according to actual needs and is not limited in this disclosure. The switching principle of the video channel is referred to the above content and will not be repeated here.

[0059] Step 205: Acquire cockpit image data based on the cockpit channel.

[0060] The cabin image data may be obtained from the cabin visual sensor based on the cabin channel, and the cabin image data may include image data or video data collected by the cabin visual sensor, which may be set according to actual needs. The cabin visual sensor may include at least one camera, such as a camera for monitoring the driver, a camera for monitoring the co-pilot user, a camera for monitoring the back row, etc., and the specific number is not limited.

[0061] Step 206: Determine a cockpit perception result based on the cockpit image data.

[0062] Among them, the perception of the cockpit can be realized based on any feasible perception algorithm or model, and the cockpit perception results include at least one of face recognition results, age and gender recognition results, phone call event recognition results, smoking event recognition results, fatigue detection results, emotion recognition results, sleep detection results, gesture recognition results and other possible perception results, which can be set according to actual needs. It can be understood that different perceptions require the use of corresponding perception algorithms or models. For example, face recognition can use a pre-trained face detection model, which can be set according to actual needs.

[0063] In an optional example, the trigger instruction includes a button trigger instruction or a voice trigger instruction. Among them, the button trigger instruction is a trigger instruction generated by the user manually triggering a hardware button or a virtual button. The hardware button is such as a parking button set on the vehicle, and the virtual button is such as a button on the screen, specifically such as a selection box on the interface in the central control screen. The voice trigger instruction is a trigger instruction generated by the user through voice interaction. For example, the user utters the voice "start parking", and the voice acquisition device collects the user's voice, determines the user's intention through voice recognition and natural language understanding, and then sends the voice trigger instruction corresponding to the user's intention to the device of the present disclosure. The specific voice interaction rules can be set according to actual needs, and the present disclosure is not limited. The present disclosure enters the parking mode through a variety of triggering methods, which effectively improves the user experience.

[0064] In an optional example, the method of the present disclosure further includes:

[0065] Step 3011, in response to receiving a trigger instruction to enter the parking mode, controlling the voice mode to switch from a multi-mode state to a single-mode state.

[0066] Among them, voice mode refers to the mode of voice recognition, single mode refers to voice recognition based only on voice data, and multi-mode refers to voice recognition combined with voice data and image data. Since entering the parking mode requires switching the video channel from the cockpit channel to the parking channel, the cockpit image data is temporarily unavailable in the cabin, so it is impossible to provide video basis for voice recognition. Single-mode voice recognition and multi-mode voice recognition require different voice recognition algorithms or models. Therefore, it is necessary to switch the voice mode to a single-mode state to use a single-mode voice recognition algorithm or model for voice recognition to ensure that the voice commands of users in the cabin can be responded to during parking. For example, during parking, users can pause and start parking through voice interaction.

[0067] In an optional example, the method of the present disclosure further includes:

[0068] Step 3021, in response to the parking mode ending, controlling the voice mode to switch from a single-mode state to a multi-mode state.

[0069] When the parking mode ends, the video channel switches from the parking channel to the cockpit channel, and the in-cabin perception is restored. In order to improve the accuracy of the in-cabin perception results, the voice mode is controlled to switch from a single-mode state to a multi-mode state, so that the user's voice and user image or video can be combined for voice recognition. Since the user's voice is easily affected by environmental noise, while the user's image is not affected by environmental noise, the user's facial or lip movements are used to assist the user's voice data in voice recognition, which can effectively improve the accuracy of voice recognition. The specific multi-mode voice recognition processing can be implemented in any feasible manner, such as based on a multi-mode voice detection model, which is not limited by this disclosure.

[0070] In an optional example, the voice function can be implemented based on full-time wake-up-free technology to ensure real-time voice control of the vehicle.

[0071] Figure 5 It is a flowchart of a time-division multiplexing method of a perception system provided by yet another exemplary embodiment of the present disclosure.

[0072] In an optional example, in response to receiving a trigger instruction to enter the parking mode, controlling the video channel to switch from the cockpit channel to the parking channel in step 201 includes:

[0073] Step 2011a, in response to receiving a trigger instruction to enter the parking mode, determining and storing current cabin occupant information based on cabin image data acquired through the cabin channel.

[0074] Specifically, after receiving the trigger command to enter the parking mode, before controlling the video channel to switch from the cockpit channel to the parking channel, the latest cockpit image data can also be obtained through the cockpit channel to perform cockpit perception, determine the current cockpit occupant information for storage, and provide a reference for parking in a narrow parking space, thereby improving the user experience. The current cockpit occupant information can at least include the co-pilot occupant information, and can also include other relevant occupant information according to actual needs, such as the rear seat occupant information, which is not specifically limited.

[0075] Step 2012a, controlling the video channel to switch from the cockpit channel to the parking channel.

[0076] Specifically, after the latest cockpit image data is collected, the video channel is switched from the cockpit channel to the parking channel to continue collecting image data outside the vehicle during the parking process, wherein the collected latest cockpit image data can provide a reference for narrow parking spaces.

[0077] In an optional example, after determining the parking perception result based on the parking image data in step 203, the method of the present disclosure further includes:

[0078] Step 204a: Determine the width type of the target parking space based on the parking perception result.

[0079] The target parking space may be the parking space to be parked that is automatically detected in the automatic parking mode, or may be the parking space to be parked by the user in the manual parking mode, and the specific details are not limited. After the target parking space is determined, the width of the target parking space may be determined based on the image captured by the camera behind the vehicle, and then the width of the target parking space may be compared with the preset parking space width threshold to determine the parking space width type. The preset parking space width threshold may be determined in combination with the width of the vehicle itself. According to the setting of the parking space width type, at least one preset parking space width threshold may be set. For example, if the parking space width type is set to a normal parking space, a narrow parking space, and an extremely narrow parking space, two different thresholds may be set, such as a first threshold and a second threshold. The first threshold is greater than the second threshold. When the width of the target parking space is less than the first threshold and greater than the second threshold, it is determined to be a narrow parking space. When the width of the target parking space is less than the second threshold, it is determined to be an extremely narrow parking space. A normal parking space means that after a vehicle is parked, the vehicle can get off normally on both sides. A narrow parking space means that after a vehicle is parked in the middle, the vehicle cannot get off normally, but can get off normally if the vehicle is offset to one side by a certain distance. An extremely narrow parking space means that no matter how the vehicle is parked, the vehicle cannot get off normally on both sides. Therefore, the parking space width type can be determined by combining the actual width of the parking space, the width of the vehicle itself, and the door opening state, and there is no specific limitation.

[0080] Step 204b: in response to the width type of the target parking space being a narrow parking space, determining the co-pilot status based on current cabin occupant information.

[0081] Among them, the co-pilot status can include occupied status and unoccupied status. When it is determined that the width type of the target parking space is a narrow parking space, the vehicle can be considered to be parked to one side so that the user in the car can get off from one side. Since the rear users can get off from both sides, the co-pilot can be used as a reference. When the co-pilot is unoccupied, it can be offset to the co-pilot side when parking, and the driving user can get off normally from the driving side; when the co-pilot is occupied, the co-pilot can be reminded to get off in advance by voice or other reminders before starting parking. Since the current cabin occupant information includes the detected co-pilot information, the co-pilot status can be easily determined. The details will not be repeated.

[0082] Step 204c, in response to the passenger side state being unoccupied, triggering parking offset to control the parking position of the vehicle to offset a preset distance toward the passenger side.

[0083] Among them, the preset distance can be determined according to the width of the target parking space and the width of the vehicle itself, ensuring that the vehicle does not exceed the target parking space after offset. For example, the width of the target parking space is 2.4 meters, the width of the vehicle itself is 2 meters, and the target parking space is a narrow parking space relative to the vehicle. With the central axis of the length direction of the target parking space and the central axis of the length direction of the vehicle as reference, when parking in the center, the central axis of the vehicle coincides with the central axis of the target parking space, leaving 0.2 meters on each side, and it may not be possible to get off normally. The preset distance can be set to 0.15 meters-0.2 meters. For example, if the vehicle is offset 0.15 meters to the co-pilot side, the driver's side can have 0.35 meters plus the parking space line, so that the driver's side can get off conveniently. The specific offset rules can be set according to actual needs, and this disclosure is not limited.

[0084] The present invention effectively improves the user experience by shifting the vehicle according to the status of the co-driver when parking in a narrow parking space, making it easier for users to get off the vehicle.

[0085] In an optional example, after step 204c, the method of the present disclosure further includes:

[0086] Step 204d, controlling the output of the disembarkation direction prompt information.

[0087] Among them, the exit direction prompt information is used to remind users to get off from the driver's side. The specific output method can be set according to actual needs. For example, a prompt voice is output through a voice output device, such as "Please get off from the left side", so that the driver and rear users can get off from the left side conveniently, further improving the user experience.

[0088] In an optional example, after determining the width type of the target parking space based on the parking perception result in step 204a, the method of the present disclosure further includes:

[0089] Step 205a: in response to the width type of the target parking space being an extremely narrow parking space, controlling output of a reminder to get off the vehicle.

[0090] Among them, an extremely narrow parking space refers to a parking space where the vehicle cannot be parked normally even if it is offset to both sides. In this case, in order to prevent the user from being unable to get off the vehicle, a reminder to get off can be controlled before parking in the target parking space to remind the user to get off first. The vehicle can be automatically parked in the parking space through remote control, further improving the user experience.

[0091] Figure 6 It is a flowchart of a time-division multiplexing method of a perception system provided by yet another exemplary embodiment of the present disclosure.

[0092] In an optional example, before controlling the video channel to switch from the cockpit channel to the parking channel in response to receiving a trigger instruction to enter the parking mode in step 201, the method of the present disclosure further includes:

[0093] Step 4011: in response to determining that a preset condition for entering the parking mode is met, controlling output of first prompt information for entering the parking mode.

[0094] Among them, the preset conditions can be set according to actual needs, such as detecting that the driver is in a fatigue state, detecting that a parking lot near the destination has been reached, etc. The first prompt information is used to prompt the driver to enter the parking mode, and the output method of the first prompt information can be set according to actual needs, such as through a prompt sound plus a central control screen display prompt, outputting a prompt voice through a voice output device, etc., without specific limitation. Exemplarily, the first prompt information is the output voice content "Has arrived at the parking lot near the destination, do you want to enter the parking mode". The specific voice content can be determined according to the actual scenario and is not limited here.

[0095] Step 4012: In response to receiving confirmation information from the user regarding the first prompt information, determining that a trigger instruction to enter the parking mode is received.

[0096] The confirmation information may be a user button confirmation or voice confirmation, which is not specifically limited. For example, if a voice prompt message is output, the user can confirm it by voice, if a prompt message is output on the central control screen, the user can click the "Confirm" or "Cancel" button on the screen page, and so on. When the user confirms the first prompt message, it is determined that a trigger instruction to enter the parking mode has been received, and a response can be made according to the aforementioned response method, which will not be repeated here.

[0097] The present disclosure can improve the user experience by promptly prompting the user to enter the parking mode when it is determined that the preset conditions for entering the parking mode are met. On the other hand, it can also effectively improve driving safety by reminding the user when the user is fatigued.

[0098] In an optional example, the method of the present disclosure further includes:

[0099] Step 4021, in the manual parking mode, in response to determining that the automatic parking conditions are met, controlling the output of automatic parking suggestion information.

[0100] Specifically, a default parking mode can be set when entering the parking mode, such as automatic parking mode or manual parking mode, and the user can switch the mode through voice or other triggering methods. It can be set specifically according to actual needs. During the parking process, when in manual parking mode, it can be detected in real time or at a fixed time whether the automatic parking conditions are met. The automatic parking conditions can be set according to actual needs. For example, if it is detected that the user has reversed and failed to park in the parking space for many times, the automatic parking suggestion information can be output to prompt the user to enter the automatic parking mode. The output of the automatic parking suggestion information can also be output in any feasible way. Please refer to the first prompt information mentioned above, and the details will not be repeated.

[0101] Step 4022, in response to receiving the user's automatic parking confirmation instruction, control switches to automatic parking mode.

[0102] When the user confirms that he can enter the automatic parking mode, he can switch to the automatic parking mode to achieve automatic parking, further improving the user experience.

[0103] In an optional example, the method of the present disclosure further includes:

[0104] Step 4031, in the automatic parking mode, in response to determining that the manual takeover conditions are met, controlling the output of manual takeover request information.

[0105] Among them, the manual takeover conditions can be set according to actual needs. For example, if an obstacle is detected and parking is impossible for a long time, a manual takeover request message can be output. The output method and content of the manual takeover request message can be set according to actual needs, such as voice output, prompt sound plus screen display, etc.

[0106] Step 4032, in response to receiving the user's manual takeover confirmation instruction, control switches to manual parking mode.

[0107] Among them, the manual takeover confirmation instruction can be triggered by any feasible method, such as detecting the user turning the steering wheel, the user inputting a confirmation voice, the user clicking a confirmation button, etc., and the details will not be repeated here.

[0108] In an optional example, in parking mode, the vehicle surrounding information can be displayed on the screen for the user based on the parking perception result, or the real-time monitoring video can be directly displayed for the user, which can be set according to actual needs. The screen display can be displayed on the screen by splicing and rendering the collected vehicle surrounding environment image data, adapting to the display frequency of the vehicle screen, and the specific display control is not repeated here.

[0109] The present disclosure provides users with a human-machine co-parking function, which enables the driving user and the vehicle to collaborate during the parking process to achieve smooth parking and further enhance the user experience.

[0110] In an optional example, the method of the present disclosure further includes:

[0111] Step 4041, in parking mode, in response to the user's voice command to pause parking, control the parking to be paused.

[0112] Specifically, during the parking process, users can pause parking at any time through voice interaction, further improving the user experience.

[0113] In an optional example, the method of the present disclosure further includes:

[0114] Step 4051, in parking mode, in response to the user's voice command to start parking, control the start of parking.

[0115] Specifically, during the parking process, after the user pauses parking, he can also start parking at any time through voice interaction, further improving the user experience.

[0116] Any of the above embodiments or optional examples of the present disclosure may be implemented separately, or may be implemented in combination in any combination without conflict.

[0117] Any time-division multiplexing method of a perception system provided in the embodiments of the present disclosure can be executed by any appropriate device with data processing capabilities, including but not limited to: terminal devices and servers, etc. Alternatively, any time-division multiplexing method of a perception system provided in the embodiments of the present disclosure can be executed by a processor, such as the processor executing any time-division multiplexing method of a perception system mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. This will not be described in detail below.

[0118] Exemplary Devices

[0119] Figure 7 is a schematic diagram of the structure of a time-division multiplexing device of a perception system provided by an exemplary embodiment of the present disclosure. The device of this embodiment can be used to implement the corresponding method embodiment of the present disclosure, such as Figure 7 The device shown includes: a first control module 501 , a first acquisition module 502 and a first processing module 503 .

[0120] The first control module 501 is used to control the video channel to switch from the cockpit channel to the parking channel in response to receiving a trigger instruction to enter the parking mode; the first acquisition module 502 is used to acquire parking image data based on the parking channel; the first processing module 503 is used to determine the parking perception result based on the parking image data acquired by the first acquisition module 502.

[0121] Figure 8 It is a structural diagram of a time-division multiplexing device of a perception system provided by another exemplary embodiment of the present disclosure.

[0122] In an optional example, the first control module 501 includes: a first control unit 5011a, used to switch the state of the cabin channel from the enabled state to the standby state, and switch the state of the parking channel from the standby state to the enabled state.

[0123] In an optional example, the apparatus of the present disclosure further includes: a second control module 601 , a second acquisition module 602 , and a second processing module 603 .

[0124] The second control module 601 is used to control the video channel to switch from the parking channel to the cockpit channel in response to the end of the parking mode; the second acquisition module 602 is used to acquire the cockpit image data based on the cockpit channel; the second processing module 603 is used to determine the cockpit perception result based on the cockpit image data acquired by the second control module 601.

[0125] In an optional example, the trigger instruction includes a key trigger instruction or a voice trigger instruction.

[0126] In an optional example, the apparatus of the present disclosure further includes: a third control module 701 and / or a fourth control module 702 .

[0127] The third control module 701 is used to control the voice mode to switch from the multi-mode state to the single-mode state in response to receiving a trigger instruction to enter the parking mode; the fourth control module 702 is used to control the voice mode to switch from the single-mode state to the multi-mode state in response to the end of the parking mode.

[0128] Fig. 9 is a structural diagram of a first control module 501 provided by an exemplary embodiment of the present disclosure.

[0129] In an optional example, the first control module 501 includes: a first processing unit 5011b and a second control unit 5012b.

[0130] The first processing unit 5011b is used to determine and store the current cabin occupant information based on the cabin image data obtained through the cabin channel in response to receiving a trigger instruction to enter the parking mode; the second control unit 5012b is used to control the video channel to switch from the cabin channel to the parking channel.

[0131] Fig.10 It is a structural diagram of a time-division multiplexing device of a perception system provided by yet another exemplary embodiment of the present disclosure.

[0132] In an optional example, the apparatus of the present disclosure further includes: a first determination module 504 , a second determination module 505 , and a third processing module 506 .

[0133] The first determination module 504 is used to determine the width type of the target parking space based on the parking perception result; the second determination module 505 is used to determine the co-pilot status based on the current cabin occupant information in response to the co-pilot status determined by the first determination module 504 as a narrow parking space; the third processing module 506 is used to trigger parking offset in response to the co-pilot status determined by the second determination module 505 as an unmanned state, so as to control the parking position of the vehicle to be offset by a preset distance toward the co-pilot side.

[0134] In an optional example, the apparatus of the present disclosure further includes:

[0135] The fifth control module 507 is used to control the output of the getting-off position prompt information.

[0136] In an optional example, the apparatus of the present disclosure further includes: a sixth control module 508, configured to control output of a reminder to get off the vehicle in response to the width type of the target parking space being an extremely narrow parking space.

[0137] Fig.11 It is a structural diagram of a time-division multiplexing device of a perception system provided by yet another exemplary embodiment of the present disclosure.

[0138] In an optional example, the apparatus of the present disclosure further includes: a seventh control module 801 and a third determination module 802 .

[0139] The seventh control module 801 is used to control the output of the first prompt information for entering the parking mode in response to determining that the preset conditions for entering the parking mode are met; the third determination module 802 is used to determine that the trigger instruction for entering the parking mode is received in response to receiving the user's confirmation information for the first prompt information.

[0140] In an optional example, the apparatus of the present disclosure further includes: an eighth control module 9011 and a ninth control module 9012 .

[0141] The eighth control module 9011 is used to control the output of automatic parking suggestion information in response to determining that the automatic parking conditions are met in the manual parking mode; the ninth control module 9012 is used to control switching to the automatic parking mode in response to receiving the user's automatic parking confirmation instruction.

[0142] In an optional example, the apparatus of the present disclosure further includes: a fourth processing module 9021 and a fifth processing module 9022 .

[0143] The fourth processing module 9021 is used to control the output of manual takeover request information in response to determining that the manual takeover conditions are met in the automatic parking mode; the fifth processing module 9022 is used to control the switch to the manual parking mode in response to receiving the user's manual takeover confirmation instruction.

[0144] In an optional example, the apparatus of the present disclosure further includes: a fifth processing module 9031 and / or a sixth processing module 9032.

[0145] The sixth processing module 9031 is used to control the pause of parking in response to the user's voice command of pause parking in the parking mode; the seventh processing module 9032 is used to control the start of parking in response to the user's voice command of start parking in the parking mode.

[0146] In an optional example, the device of the present disclosure can also be divided into modules in any other feasible manner, such as being divided into an operation processing module, a perception regulation and control module, and a video signal scheduling module, wherein the video signal scheduling module is used for switching video channels, the operation processing module is used for acquiring cockpit image data based on the cockpit channel or acquiring parking image data based on the parking channel, and performing perception processing based on the acquired image data to obtain perception results, and the perception regulation and control module is used for planning and controlling based on the perception results of the operation processing module, such as parking control, display control, and so on. The operation processing module can be any module used for operation processing, such as an AI operation processing unit, an AI chip, or other possible processing modules. It can be specifically set according to actual needs.

[0147] Exemplary Electronic Devices

[0148] The embodiment of the present disclosure also provides an electronic device, including: a memory for storing a computer program;

[0149] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the time-division multiplexing method of the perception system described in any of the above embodiments of the present disclosure is implemented.

[0150] Fig.12 FIG. 1 is a schematic diagram of a structure of an application embodiment of the electronic device disclosed in the present invention. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.

[0151] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0152] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the methods of the various embodiments of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0153] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0154] For example, the input device 13 may be the microphone or microphone array mentioned above, which is used to capture the input signal of the sound source.

[0155] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0156] The output device 14 can output various information to the outside, including the determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0157] Of course, to simplify, Fig.12 Only some of the components related to the present disclosure in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0158] Exemplary computer program products and computer-readable storage media

[0159] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0160] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0161] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0162] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0163] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0164] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0165] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0166] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0167] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0168] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0169] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A time-division multiplexing method for a perception system, comprising: In response to receiving a trigger instruction to enter a parking mode, controlling the video channel to switch from a cockpit channel to a parking channel; acquiring parking image data based on the parking channel; determining a parking perception result based on the parking image data; The parking perception result includes a parking space detection result and an obstacle recognition result; Wherein, in response to receiving a trigger instruction to enter the parking mode, controlling the video channel to switch from the cockpit channel to the parking channel includes: In response to receiving the trigger instruction to enter the parking mode, determining and storing current cabin occupant information based on cabin image data acquired through the cabin channel; Controlling the video channel to switch from the cockpit channel to the parking channel; After determining a parking perception result based on the parking image data, the method further includes: Based on the parking perception result, determining the width type of the target parking space; In response to the width type of the target parking space being a narrow parking space, determining a co-pilot state based on the current cabin occupant information; In response to the passenger vehicle state being an unmanned state, triggering parking shifting to control the parking position of the vehicle to shift toward the passenger vehicle side by a preset distance; The method further includes: in response to receiving the trigger instruction to enter the parking mode, controlling the voice mode to switch from a multi-mode state to a single-mode state; the single-mode state refers to a state in which voice recognition is performed based only on voice data; the multi-mode state refers to a state in which voice recognition is performed based on voice data and image data; The method further comprises: In response to the parking mode being ended, controlling the video channel to switch from the parking channel to the cockpit channel; Acquiring cockpit image data based on the cockpit channel; Determine a cockpit perception result based on the cockpit image data; the cockpit perception result includes at least one of a face recognition result, an age and gender recognition result, a phone call event recognition result, a smoking event recognition result, a fatigue detection result, an emotion recognition result, a sleep detection result, and a gesture recognition result; Among them, cabin perception and parking perception share a domain controller.

2. The method according to claim 1, wherein: The control video channel is switched from the cockpit channel to the parking channel, including: The state of the cockpit passage is switched from the enabled state to the standby state, and the state of the parking passage is switched from the standby state to the enabled state.

3. The method according to claim 1, wherein: The trigger instruction includes a key trigger instruction or a voice trigger instruction, and the method further includes: In response to the parking mode being ended, the voice mode is controlled to switch from the single mode state to the multi-mode state.

4. The method according to claim 1, wherein: After triggering the parking deviation, the method further includes: Control the output of the vehicle disembarkation direction prompt information.

5. The method according to claim 1, wherein: After determining the width type of the target parking space based on the parking perception result, the method further includes: In response to the width type of the target parking space being an extremely narrow parking space, the control outputs a reminder to get off the vehicle.

6. The method according to claim 1, wherein: In response to receiving a trigger instruction for entering the parking mode, before controlling the video channel to switch from the cockpit channel to the parking channel, the method further includes: In response to determining that a preset condition for entering the parking mode is met, controlling output of first prompt information for entering the parking mode; In response to receiving confirmation information of the user in response to the first prompt information, it is determined that a trigger instruction for entering the parking mode is received.

7. The method according to any one of claims 1 to 6, further comprising: In the manual parking mode, in response to determining that the automatic parking condition is satisfied, controlling output of automatic parking suggestion information; In response to receiving a user's automatic parking confirmation command, controlling switching to an automatic parking mode; or, In the automatic parking mode, in response to determining that a manual takeover condition is met, controlling output of manual takeover request information; In response to receiving a manual takeover confirmation instruction from the user, control is switched to the manual parking mode.

8. The method according to any one of claims 1 to 6, further comprising: In the parking mode, in response to a user's voice command for pausing parking, controlling the pausing parking; And / or, in response to a voice command of the user to start parking, control the start of parking.

9. A time-division multiplexing device for a sensing system, comprising: A first control module, configured to control the video channel to switch from the cockpit channel to the parking channel in response to receiving a trigger instruction to enter the parking mode; A first acquisition module, configured to acquire parking image data based on the parking channel; A first processing module, configured to determine a parking perception result based on the parking image data; The parking perception result includes a parking space detection result and an obstacle recognition result; Wherein, the first control module includes: a first processing unit, configured to determine and store current cabin occupant information based on cabin image data acquired through the cabin channel in response to receiving the trigger instruction to enter the parking mode; a second control unit, configured to control a video channel to switch from the cockpit channel to the parking channel; The device also includes: A first determination module, configured to determine a width type of a target parking space based on the parking perception result; a second determination module, configured to determine the co-pilot status based on the current cabin occupant information in response to the width type of the target parking space being a narrow parking space; A third processing module, configured to trigger parking shift in response to the passenger vehicle state being an unmanned state, so as to control the parking position of the vehicle to shift toward the passenger vehicle side by a preset distance; a third control module, configured to control the voice mode to switch from a multi-mode state to a single-mode state in response to receiving the trigger instruction to enter the parking mode; the single-mode state refers to a state in which voice recognition is performed based only on voice data; the multi-mode state refers to a state in which voice recognition is performed based on voice data and image data; a second control module, configured to control a video channel to switch from the parking channel to the cockpit channel in response to the parking mode being terminated; A second acquisition module, configured to acquire cockpit image data based on the cockpit channel; a second processing module, for determining a cockpit perception result based on the cockpit image data; the cockpit perception result includes at least one of a face recognition result, an age and gender recognition result, a phone call event recognition result, a smoking event recognition result, a fatigue detection result, an emotion recognition result, a sleep detection result, and a gesture recognition result; Among them, cabin perception and parking perception share a domain controller.

10. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the time-division multiplexing method of the perception system described in any one of claims 1 to 8.

11. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the time-sharing multiplexing method of the perception system described in any one of claims 1-8.

Images