Vehicle visual identification and switch sticker interaction method and identification system

Abstract

The invention relates to the technical field of intelligent cabins, and discloses a visual identification and switch sticker interaction method and identification system for a vehicle, and the method comprises the steps: a visual module which carries out the image collection of a preset switch sticker through employing at least one camera, and determines a switch sticker image; the central control module is used for performing feature extraction on the switch sticker image by adopting an image algorithm, identifying whether the switch sticker is manually pressed in the switch sticker image, and sending a control instruction to the execution module when determining that the switch sticker is manually pressed; and the execution module receives the control instruction from the central control module, and executes a preset operation corresponding to the switch sticker based on the control instruction. According to the method provided by the embodiment of the invention, the visual module and the switch sticker are utilized, so that a traditional physical button and a complex wire harness system are avoided, the number and complexity of equipment in the vehicle are reduced, and the hardware cost of the vehicle is reduced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of intelligent cockpits, and particularly to a visual recognition and switch sticker interaction method and recognition system for a vehicle. Background Art

[0002] With the rapid development of intelligent cockpit technology, the number of vehicle control devices inside a vehicle is also increasing rapidly.

[0003] In related technologies, vehicle control devices often still rely on the control of physical devices and wiring harnesses inside the vehicle, resulting in an increase in the control cost of vehicle condition devices, and thus the installation cost of the vehicle also correspondingly increases. Therefore, how to reduce the control cost of vehicle condition devices and thus reduce the installation cost of the vehicle has become a problem to be solved. Summary of the Invention

[0004] In view of this, the present disclosure provides a visual recognition and switch sticker interaction method and recognition system for a vehicle to solve the problem of how to reduce the control cost of vehicle condition devices and thus reduce the installation cost of the vehicle.

[0005] On the one hand, the present disclosure provides a visual recognition and switch sticker interaction method for a vehicle, which is applied to a recognition system. The recognition system includes: a visual module, a central control module, and an execution module. The method includes: the visual module uses at least one camera to collect an image of a preset switch sticker to determine a switch sticker image; the central control module uses an image algorithm to extract features from the switch sticker image, and identifies whether the switch sticker is manually pressed in the switch sticker image. When it is determined that the switch sticker is manually pressed, a control instruction is sent to the execution module; the execution module receives the control instruction from the central control module and executes a preset operation corresponding to the switch sticker based on the control instruction.

[0006] On the other hand, the present disclosure also provides a recognition system, which includes a visual module, a central control module, and an execution module. Among them: the visual module is used to use at least one camera to collect an image of a preset switch sticker to determine a switch sticker image; the central control module is used to use an image algorithm to extract features from the switch sticker image, and identify whether the switch sticker is manually pressed in the switch sticker image. When it is determined that the switch sticker is manually pressed, a control instruction is sent to the execution module; the execution module is used to receive the control instruction from the central control module and execute a preset operation corresponding to the switch sticker based on the control instruction.

[0007] On the other hand, the present disclosure also provides a computer-readable storage medium, on which computer instructions are stored. The computer instructions are used to cause a computer to implement the above-mentioned visual recognition and switch sticker interaction method for a vehicle.

[0008] On the other hand, the present disclosure also provides a computer program product, including computer instructions for causing a computer to execute the above-mentioned vehicle visual recognition and switch sticker interaction method.

[0009] Through the vehicle visual recognition and switch sticker interaction method and recognition system of the above embodiments of the present disclosure, by using the visual module and the switch sticker, the traditional physical buttons and complex wiring harness systems are avoided, the number and complexity of in-vehicle devices are reduced, and the hardware cost of the vehicle is lowered.

[0010] In addition, by using an image algorithm to recognize whether the switch sticker is pressed, a more intelligent and intuitive interaction method can be realized, enabling the vehicle owner or passenger to control vehicle functions simply by a simple gesture or touch, thus enhancing the interactivity and convenience of the cockpit. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the related art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the related art. Obviously, the drawings in the following description are some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1a FIG. shows an exemplary schematic diagram of the architecture of a recognition system to which a vehicle visual recognition and switch sticker interaction method according to an embodiment of the present disclosure is applied;

[0013] Figure 1b is a flowchart of a vehicle visual recognition and switch sticker interaction method provided by an embodiment of the present disclosure;

[0014] Figure 2a is a first schematic diagram of hand feature detection in a vehicle visual recognition and switch sticker interaction method provided by an embodiment of the present disclosure;

[0015] Figure 2b is a second schematic diagram of hand feature detection in a vehicle visual recognition and switch sticker interaction method provided by an embodiment of the present disclosure;

[0016] Figure 3 FIG. shows a preset flowchart of a vehicle visual recognition and switch sticker interaction method according to an embodiment of the present disclosure;

[0017] Figure 4 FIG. shows a recognition flowchart of a vehicle visual recognition and switch sticker interaction method according to an embodiment of the present disclosure;

[0018] Figure 5 is a schematic diagram of the structure of another recognition system provided by an embodiment of the present disclosure;

[0019] Figure 6 It is a schematic structural diagram of another recognition system provided by an embodiment of the present disclosure. Detailed implementation manners

[0020] With the continuous development of automotive technology, the intelligence level of vehicles has been continuously improved, especially the application of intelligent cockpits has become more and more widespread. In modern vehicles, multimodal perception technology has become an important part of enhancing the user experience. To make vehicles more intelligent, a variety of perception devices are integrated in the vehicle, such as microphone arrays, cameras, sensors, etc. These devices cooperate with the control system to realize the perception and interaction of the vehicle interior environment, so as to provide a more personalized and convenient experience for vehicle owners.

[0021] Currently, in related technologies, various functional control systems in the vehicle usually rely on devices such as physical buttons, switch stickers or touch screens to achieve operations. These control systems can be divided into two categories:

[0022] Retrofit flexible sticker solution: This solution uses wireless technology to connect the switch sticker to the receiving end. The switch sticker is built-in with a wireless signal transmission module. When the user touches or presses the switch, a fixed signal is transmitted to the receiving end, and the receiving end executes preset operations according to this signal, such as controlling the lights, air conditioner in the cockpit or turning on a specific driving mode, etc. The advantage of this solution is that it is easy to install and does not require too many wiring harnesses and power supplies. However, its disadvantage is that it requires an additional wireless communication module and battery power supply, increasing the cost and energy efficiency problems.

[0023] Factory-installed button solution: In this solution, the vehicle is equipped with physical buttons (such as air conditioner control buttons, reading lamp buttons, etc.), and is connected to the Controller Area Network (CAN) or Local Interconnect Network (LIN) bus in the vehicle through capacitive switch technology. The pressing action of the button is transmitted to the vehicle machine system through capacitive induction, and then corresponding control instructions are triggered. The advantage of this solution is that the operation is simple and stable, but there are certain disadvantages, including the need for complex wiring harness connections, high installation costs, and possible misoperations caused by quality problems of capacitive buttons.

[0024] In related technologies, the following problems often exist: For the retrofit flexible sticker solution, first, the switch needs power supply; second, a wireless communication module is required between the switch and the central control; third, the button needs a capacitive switch. For the factory-installed button solution, first, a CAN / LIN wiring harness is required for communication and power supply between the physical button and the central control; second, the physical button needs a capacitive switch. Therefore, there are problems of high installation costs and high complexity in the prior art.

[0025] To solve the above problems, various embodiments of the present disclosure provide a method for visual recognition of a vehicle and interaction with a switch sticker, which is applied to an identification system. The identification system includes: a visual module, a central control module, and an execution module. The method includes: the visual module uses at least one camera to collect an image of a preset switch sticker to determine a switch sticker image; the central control module uses an image algorithm to extract features from the switch sticker image, and identifies whether the switch sticker is manually pressed in the switch sticker image. When it is determined that the switch sticker is manually pressed, a control instruction is sent to the execution module; the execution module receives the control instruction from the central control module and executes a preset operation corresponding to the switch sticker based on the control instruction.

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Apparently, the described embodiments are some, but not all, of the embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

[0027] Please refer to Figure 1a , Figure 1a which shows an exemplary schematic diagram of the architecture of an identification system to which a method for visual recognition of a vehicle and interaction with a switch sticker according to an embodiment of the present disclosure is applied. As Figure 1a shown, the identification system may include: a visual module, a central control module, and an execution module.

[0028] As Figure 1a shown, the identification system may be a system integrating multiple modules, which is used to identify the switch sticker inside the vehicle and its state through visual perception, and trigger a preset operation according to the recognition result.

[0029] Among them, the visual module in the identification system may be composed of at least one camera, and may be responsible for image collection and image recognition; the central control module in the identification system may be responsible for receiving image data from the visual module and processing and analyzing the image. The central control module may include at least one of the following: a processor, a memory, and an algorithm module; the execution module in the identification system may control the actual devices inside the vehicle to respond according to the instructions of the central control module.

[0030] Further referring to Figure 1b , Figure 1b is a flowchart of a method for visual recognition of a vehicle and interaction with a switch sticker provided by an embodiment of the present disclosure, which is applied to the identification system shown in the above Figure 1a . The process of the method may include the following steps:

[0031] Step S101: The vision module uses at least one camera to collect images of a preset switch sticker and determines the switch sticker image.

[0032] In this embodiment, the preset position of the switch sticker in the vehicle can take into account the recognition angle and difficulty of the vision module, reduce the interference of environmental factors on recognition, and ensure that the camera can clearly collect the image of the switch sticker. For example, the preset position of the switch sticker can include, but is not limited to: the roof of the vehicle, the vehicle center console, the door control area, etc.

[0033] Here, the preset switch sticker can refer to a visual identifier pre-designed and installed at a specific position in the vehicle cockpit, which has no circuit connection or signal transmission with the vehicle.

[0034] As an example, the switch sticker can be located on the vehicle center console. When the switch sticker is pressed, it can be used to adjust the air conditioning temperature. Among them, the front part of the switch sticker can carry text prompts to enhance the function of the current switch sticker for the user. The switch sticker can also have a backlight design on the front part to improve the user's convenience of use in low visibility driving scenarios.

[0035] In a possible implementation, the vision module periodically collects the switch sticker image based on a preset time. The switch sticker image can include: real-time video frames and / or static images;

[0036] The vision module formats and preprocesses the collected switch sticker image to ensure that the image data meets the format requirements of the image algorithm in the central control module;

[0037] The vision module sends the formatted and preprocessed switch sticker image to the central control module.

[0038] Step S102: The central control module uses an image algorithm to extract features from the switch sticker image, identifies whether the switch sticker is manually pressed in the switch sticker image, and sends a control instruction to the execution module when it is determined that the switch sticker is manually pressed.

[0039] In this embodiment, before the central control module uses the image algorithm to extract features from the switch sticker image, it further includes:

[0040] The central control module removes noise and / or interference in the switch sticker image, and converts the switch sticker image after removing noise and / or interference from a color image to a grayscale image for easy feature extraction and analysis;

[0041] The central control module ensures the alignment of the image by performing image correction on the switch sticker image.

[0042] In a possible implementation, the central control module extracts features from the switch sticker image using image algorithms. Feature extraction may include, but is not limited to, texture features and shape features.

[0043] Among them, the texture features can be used by a filter to extract the texture features in the switch sticker image and distinguish the details in the switch sticker pattern; the shape features can be obtained by identifying the edges, shapes or geometric features of the switch sticker.

[0044] Further, identifying whether the switch sticker is manually pressed in the switch sticker image may include:

[0045] The central control module analyzes the differences in the switch sticker image by comparing the images collected before and after. If the occluded feature of the switch sticker is greater than a preset threshold, it is determined that the switch sticker is manually pressed.

[0046] Furthermore, the central control module can send control instructions to the execution module using an in-vehicle network. Among them, the in-vehicle network may include, but is not limited to, CAN, LIN, and Ethernet.

[0047] Among them, the control instructions may include: the switch sticker identifier and the preset operation corresponding to the switch sticker.

[0048] In a possible implementation, the method may further include: the temperature sensor in the vehicle-mounted sensor obtains the temperature feature of the switch sticker area, and the central control module judges the temperature change before and after the switch sticker area to determine whether there is a temperature rise in the switch sticker caused by the user pressing the switch sticker, so as to judge whether the switch sticker is manually pressed.

[0049] Step S103, the execution module receives the control instruction from the central control module and executes the preset operation corresponding to the switch sticker based on the control instruction.

[0050] In this embodiment, the execution module receives the control instruction from the central control module, parses the control instruction, and extracts the target device and operation type in the control instruction. Among them, the target device may refer to the device to be controlled, and the operation type may refer to the specific action to be executed.

[0051] Further, the execution module queries the current state of the target device to avoid repeated execution of operations on the target device. For example, when the target device is an air conditioner, if the air conditioner is already turned on, there is no need to execute the turn-on operation again. At this time, the execution module discards the current control instruction.

[0052] Furthermore, when the current state of the target device meets the requirements, the execution module sends the corresponding operation type to the target device, enables the target device to execute the operation corresponding to the operation type, and obtains the operation result of the target device executing the operation.

[0053] In the vehicle visual recognition and switch sticker interaction method and recognition system of the above embodiments of the present disclosure, by using the visual module and the switch sticker, the traditional physical buttons and complex wiring harness systems are avoided, the number and complexity of in-vehicle devices are reduced, not only the hardware cost is reduced, but also the design of the in-vehicle control system is simplified, making the interior space layout of the vehicle more concise. Using image algorithms to recognize whether the switch sticker is pressed can achieve a more intelligent and intuitive interaction method. It enables the vehicle owner or passenger to control vehicle functions with simple gestures or touches, improving the interactivity and convenience of the cockpit. Through feature extraction and recognition of manual pressing by image algorithms, it can more accurately determine whether the switch sticker is pressed, avoiding the accidental touch and unstable physical contact problems that may occur with traditional buttons, thereby improving the reliability of the intelligent cockpit system.

[0054] In a possible implementation manner of the above embodiment, the switch sticker includes at least one of the following: a physical sticker with a preset feature pattern or a feature point at a preset position of the vehicle.

[0055] In this embodiment, the physical sticker with a preset feature pattern may refer to a physical patch with a unique and recognizable pattern. The preset adjustment pattern carried by each physical sticker may be unique and correspond one-to-one to the in-vehicle device to be controlled.

[0056] As an example, the preset adjustment pattern carried by each physical sticker may include, but is not limited to: two-dimensional code, bar code, circle, triangle, etc. Among them, the circle may correspond to adjusting the air conditioner temperature, and the triangle may correspond to adjusting the volume.

[0057] Furthermore, the feature point at a preset position of the vehicle may be an area marked by specific processing during the vehicle design and manufacturing stage. The feature point may have unique visual features and can be captured by the camera and interact with functions. The setting of the feature point does not require the user to manually add additional stickers or labels.

[0058] As an example, there is a concave oval dent on the edge of the vehicle's center console. When the user covers this area with a finger, the trunk can be opened.

[0059] In a possible implementation manner, the corresponding association relationship between different styles of switch stickers and in-vehicle devices can be custom-set by the user.

[0060] In the vehicle visual recognition and switch sticker interaction method and recognition system of the above embodiments of the present disclosure, by providing switch stickers in various forms, users can flexibly select according to their needs, improving the flexibility of vehicle cockpit system interaction. There is no need to provide power supply and circuit connection for the switch stickers, and it completely relies on the algorithms of the visual module and the central control module for judgment, avoiding the dependence on power supply and communication modules in traditional solutions. At the same time, it saves the production, installation and wiring harness costs of physical switches, reduces the risk of mechanical failures, and extends the service life of the system. The physical stickers can be pasted at the positions commonly used by users to meet the needs of habitual operations and improve the personalization level of the vehicle.

[0061] In a possible implementation manner of the above embodiment, the method further includes:

[0062] The visual module, when the first switch sticker is a physical sticker with a preset feature pattern, after the first switch sticker is installed, uses at least one camera to collect an image of the first switch sticker to determine the first switch sticker image;

[0063] The central control module, based on the first switch sticker image, determines the first coordinate of the first switch sticker and recognizes the preset feature pattern in the first switch sticker image;

[0064] The central control module, based on the preset feature pattern, associates the first switch sticker with a preset operation of the execution module to determine the first association relationship, and saves the preset feature pattern, the first coordinate and the first association relationship;

[0065] The visual module, when the second switch sticker is a feature point at a preset position of the vehicle, uses at least one camera to collect an image of the feature point at the preset position of the vehicle to determine the second switch sticker image;

[0066] The central control module, based on the second switch sticker image, determines the second coordinate of the second switch sticker and recognizes the feature point in the second switch sticker image;

[0067] The central control module, based on the feature point, associates the second switch sticker with a preset operation of the execution module to determine the second association relationship, and saves the feature point, the second coordinate and the second association relationship.

[0068] In this embodiment, the central control module, based on the first switch sticker image, determines the first coordinate of the first switch sticker and recognizes the preset feature pattern in the first switch sticker image, which may include:

[0069] After receiving the first switch sticker image from the visual module, the central control module uses an image algorithm to extract the preset feature pattern of the first switch sticker and determines the binary coordinate of the first switch sticker as the first coordinate.

[0070] Further, the central control module associates the first switch sticker with the preset operation of the execution module based on a preset feature pattern to determine the first association relationship, which may include:

[0071] The central control module obtains the preset operation corresponding to the preset feature pattern based on the user's custom information, associates the preset feature pattern with the preset operation to determine the first association relationship, and stores the first association relationship in a preset mapping table.

[0072] Among them, the preset mapping table can be used to bind the switch sticker features collected by the vision module to the operations corresponding to the vehicle function module. When the user presses the switch sticker, the central control module can quickly match the function of the switch sticker based on the preset mapping table, so that the execution module executes the corresponding preset operation.

[0073] As an example, the content of the preset mapping table can be as shown in Table 1 below:

[0074] Preset feature pattern First coordinate Preset operation Circle (1,2) Turn on the reading light Star (1.5,50) Adjust the driving mode First barcode (100,60) Adjust the seat position

[0075] Table 1

[0076] In a possible implementation, the user can associate the switch codes of different patterns with the cockpit functions through the vehicle's central control screen or in-vehicle application; the central control module receives the custom information input by the user, binds the pattern to the preset operation of the cockpit function, and updates the preset mapping table.

[0077] The association and update steps of the second switch sticker in the preset mapping table are the same as those of the first switch sticker, which will not be elaborated here.

[0078] As an example, the content of the preset mapping table after updating the second switch sticker can be as shown in Table 2 below:

[0079] Preset feature pattern First coordinate Preset operation Circle (1,2) Turn on the reading light Star (1.5,50) Adjust the driving mode First barcode (100,60) Adjust the seat position Feature point 1 (500,600) Turn on the air conditioner Feature point 2 (400,60) Adjust the audio mode

[0080] Table 2

[0081] In the vehicle visual recognition and switch sticker interaction method and recognition system of the above embodiments of the present disclosure, the connection between the switch sticker and the receiving end can be more convenient, and the logic is closed-loop through the upper-layer application of the cockpit. There is no need to transmit signals through complex communication protocols, so there is no need to open up the driver layer of wireless signals, making the software more general. The method only relies on the existing cameras and visual modules in the vehicle, without the need for additional wireless signal transmission modules or complex physical buttons, reducing the hardware procurement and installation costs. At the same time, it avoids large-scale transformation of the existing vehicle wiring system, further saving the manufacturing cost. The method binds the pattern (or feature points) of each switch sticker to specific preset operations through a mapping table, which can be updated or adjusted at any time, facilitating later maintenance and function expansion. When the vehicle function changes, only the mapping table needs to be updated at the software level, without the need to replace the hardware.

[0082] In a possible implementation manner of the above step S102, the central control module uses an image algorithm to extract features from the switch sticker image and identify whether the switch sticker is manually pressed in the switch sticker image, which is implemented based on the following steps:

[0083] The central control module determines whether the switch sticker image is blocked. When the switch sticker image is blocked, the switch sticker image is enlarged to obtain the enlarged switch sticker image.

[0084] The central control module determines whether the enlarged switch sticker image is completely blocked based on the blocked area of the preset feature pattern or feature points in the enlarged switch sticker image.

[0085] When the enlarged switch sticker image is not completely blocked, the central control module identifies whether there is a hand feature in the enlarged switch sticker image. When it is determined that there is a hand feature, it is determined that the switch sticker corresponding to the switch sticker image is pressed by a finger.

[0086] When the enlarged switch sticker image is completely blocked, the central control module identifies whether there is a hand feature in the enlarged switch sticker image. When it is determined that there is a hand feature, it is determined that the switch sticker corresponding to the switch sticker image is pressed by a finger.

[0087] In this embodiment, the central control module determines whether the switch sticker image is blocked. When the switch sticker image is blocked, the switch sticker image is enlarged to obtain the enlarged switch sticker image, which may include:

[0088] The central control module receives the original switch sticker image from the visual module, uses an image processing algorithm to locate the switch sticker area, and extracts the preset feature pattern or specific feature points in the switch sticker area.

[0089] The central control module determines whether there is a missing area in the switch sticker area based on a preset characteristic pattern or specific feature points. If there is a missing area, it determines that the switch sticker image is occluded, and further determines whether the occluded area of the switch sticker area is greater than a preset occlusion threshold. When the occluded area is greater than the preset occlusion threshold, an image enlargement process is performed on the switch sticker image to determine the enlarged switch sticker image.

[0090] Here, the occluded area can refer to the proportion of the occluded part in the total area of the switch sticker image. When the occluded area is greater than the preset occlusion threshold, it can be determined that the switch sticker image is severely occluded.

[0091] Furthermore, the image enlargement process can refer to the process of expanding or filling certain areas of the image in order to restore or enhance the image information. In this embodiment, the image enlargement process can be used when the switch sticker image is occluded. By expanding the boundary area of the image, the occluded part can be better restored to ensure that the visual recognition system can continue to accurately recognize the key information of the image.

[0092] The central control module determines whether the enlarged switch sticker image is completely occluded based on the occluded area of the preset characteristic pattern or feature points in the enlarged switch sticker image, which may include:

[0093] The central control module determines that the enlarged switch sticker image is completely occluded when it determines that the occluded area is 100%.

[0094] Here, complete occlusion can mean that the enlarged switch sticker image is completely covered by other objects, so that no recognizable information can be obtained.

[0095] Furthermore, when the enlarged switch sticker image is not completely occluded, the central control module uses an image algorithm to identify whether there is a hand feature in the enlarged switch sticker image. When it determines that there is a hand feature, it determines that the switch sticker corresponding to the switch sticker image is pressed by a finger;

[0096] When the enlarged switch sticker image is completely occluded, the central control module uses an image algorithm to identify whether there is a hand feature in the enlarged switch sticker image. When it determines that there is a hand feature, it determines that the switch sticker corresponding to the switch sticker image is pressed by a finger.

[0097] As an example, the case where the enlarged switch sticker image is not completely occluded can be as Figure 2a shown, Figure 2a is the first schematic diagram of hand feature detection in a visual recognition and switch sticker interaction method for a vehicle provided by an embodiment of the present disclosure. Among them, when the switch sticker is not completely occluded, an image enlargement process can be performed on the switch sticker to detect whether there is a hand feature in the enlarged switch sticker image. When it is detected that there is a hand feature, it is determined that the switch sticker is manually pressed. When it is detected that there is no hand feature, it is determined that the switch sticker is not manually pressed.

[0098] Similarly, the case where the enlarged image of the switch sticker is completely blocked can be as Figure 2b shown Figure 2b This is the second schematic diagram of hand feature detection for a visual recognition and switch sticker interaction method of a vehicle provided by an embodiment of the present disclosure. When the switch sticker is completely blocked, an enlarged image of the switch sticker is processed, and whether there are hand features in the enlarged image of the switch sticker is detected. When it is detected that there are hand features, it is determined that the switch sticker is manually pressed. When it is detected that there are no hand features, it is determined that the switch sticker is not manually pressed.

[0099] In the vehicle visual recognition and switch sticker interaction method and recognition system of the above embodiments of the present disclosure, even if the switch sticker image is blocked, the system can still accurately recognize the pressing action according to the enlarged image, avoiding misjudgment caused by limited viewing angle or occlusion. Further, even in the case of complete occlusion, the pressing action can be judged by detecting hand contours, shapes or gesture features, enhancing the robustness and application range of the system. Especially when the user covers the switch sticker, the operation intention can still be correctly judged.

[0100] In a possible implementation manner of the above embodiment, the method further includes:

[0101] The central control module receives at least one switch sticker image from the visual module, and the switch sticker image is obtained by the at least one camera in the visual module collecting the preset switch sticker from multiple angles;

[0102] The central control module uses differential calculation to determine the confidence level corresponding to each switch sticker image based on the image change situation in the at least one switch sticker image, and determines whether the switch sticker is pressed by a finger based on the at least one confidence level.

[0103] In this embodiment, the visual module calls at least one camera to collect the preset switch sticker from multiple angles to obtain multiple images including the switch sticker.

[0104] The central control module performs frame-by-frame differential calculation on the multi-angle images, calculates the pixel changes between two frames or multiple frames of images to judge the dynamic changes of the switch sticker image; the central control module matches the image change area obtained by the differential calculation with the preset switch sticker pattern or feature points to determine whether the change occurs in the switch sticker area.

[0105] The central control module evaluates the confidence level of finger pressing according to the differential calculation results in each switch sticker image, integrates multiple confidence levels, and determines whether the switch sticker is pressed by a finger based on the integrated final confidence level.

[0106] In the vehicle vision recognition and switch sticker interaction method and recognition system of the above embodiments of the present disclosure, multi-angle image data is fully utilized to reduce misjudgment that may be caused by a single angle. Even when the switch sticker is under partial occlusion, light changes or other interference conditions, it is still possible to accurately determine whether the switch sticker is pressed based on differential calculation and confidence analysis, improving the robustness of recognition.

[0107] In a specific embodiment, Figure 3 The preset process schematic diagram of a vehicle vision recognition and switch sticker interaction method according to an embodiment of the present disclosure is shown. This process includes the following steps:

[0108] Step S301, take pictures of the interior of the vehicle cabin;

[0109] Here, the vision module takes the first switch sticker image and the second switch sticker image;

[0110] Step S302, recognize the image features of the switch sticker in the picture;

[0111] Here, the central control module recognizes the first coordinate and the preset feature pattern of the first switch sticker, and the second coordinate and feature points of the second switch sticker;

[0112] Step S303, set the corresponding relationship between the switch sticker and the in-vehicle device;

[0113] Here, the central control module associates the first switch sticker with the preset operation of the execution module based on the preset feature pattern to determine the first association relationship, and associates the second switch sticker with the preset operation of the execution module based on the feature points to determine the second association relationship;

[0114] Step S304, save the image features, binary coordinates, and corresponding relationships;

[0115] Here, the central control module saves the preset feature pattern, the first coordinate and the first association relationship, and saves the feature points, the second coordinate and the second association relationship.

[0116] In a specific embodiment, Figure 4 The recognition process schematic diagram of a vehicle vision recognition and switch sticker interaction method according to an embodiment of the present disclosure is shown. This process includes the following steps:

[0117] Step S401, take pictures of the interior of the vehicle cabin;

[0118] Here, the central control module acquires the first switch sticker image and the second switch sticker image;

[0119] Step S402, determine whether the switch sticker image is occluded; if so, go to step S403, if not, go to step S405;

[0120] Step S403, determine whether the switch sticker is pressed; if so, proceed to step S404, if not, proceed to step S405;

[0121] Here, perform an enlargement process on the switch sticker image, and detect whether there are hand features in the enlarged switch sticker image. If there are, determine that the switch sticker is pressed;

[0122] Step S404, perform a preset operation;

[0123] Step S405, do not perform the preset operation.

[0124] In one embodiment, an identification system 500 is provided. The identification system 500 corresponds one-to-one with the vehicle visual recognition and switch sticker interaction method in the above embodiment. As Figure 5 shown, the identification system 500 includes a vision module 501, a central control module 502, and an execution module 503. Among them, the detailed descriptions of each functional module are as follows:

[0125] The vision module 501 is used to collect an image of a preset switch sticker by using at least one camera to determine the switch sticker image;

[0126] The central control module 502 is used to extract features from the switch sticker image by using an image algorithm, identify whether the switch sticker is manually pressed in the switch sticker image, and send a control instruction to the execution module when it is determined that the switch sticker is manually pressed;

[0127] The execution module 503 is used to receive the control instruction from the central control module and perform the preset operation corresponding to the switch sticker based on the control instruction.

[0128] In one embodiment, the switch sticker includes at least one of the following: a physical sticker with a preset feature pattern or a feature point at a preset position of the vehicle.

[0129] In one embodiment, the vision module 501 is further used to, when the first switch sticker is a physical sticker with a preset feature pattern, collect an image of the first switch sticker by using at least one camera after the first switch sticker is installed to determine the first switch sticker image;

[0130] The central control module 502 is used to determine the first coordinate of the first switch sticker based on the first switch sticker image and identify the preset feature pattern in the first switch sticker image;

[0131] The central control module 502 is used to associate the first switch sticker with the preset operation of the execution module based on the preset feature pattern, determine the first association relationship, and save the preset feature pattern, the first coordinate, and the first association relationship;

[0132] The vision module 501 is used to collect images of the feature points at the preset position of the vehicle by using at least one camera when the second switch sticker is at the feature points at the preset position of the vehicle, and determine the second switch sticker image;

[0133] The central control module 502 is used to determine the second coordinates of the second switch sticker based on the second switch sticker image, and identify the feature points in the second switch sticker image;

[0134] The central control module 502 is used to associate the second switch sticker with the preset operation of the execution module based on the feature points, determine the second association relationship, and save the feature points, the second coordinates and the second association relationship.

[0135] In one embodiment, the central control module 502 is used to determine whether the switch sticker image is blocked. When the switch sticker image is blocked, perform an image enlargement process on the switch sticker image to determine the enlarged switch sticker image;

[0136] The central control module 502 is used to determine whether the enlarged switch sticker image is completely blocked based on the preset feature pattern or the occlusion area of the feature points in the enlarged switch sticker image;

[0137] The central control module 502 is used to identify whether there is a hand feature in the enlarged switch sticker image when the enlarged switch sticker image is not completely blocked. When it is determined that there is a hand feature, it is determined that the switch sticker corresponding to the switch sticker image is pressed by a finger;

[0138] The central control module 502 is used to identify whether there is a hand feature in the enlarged switch sticker image when the enlarged switch sticker image is completely blocked. When it is determined that there is a hand feature, it is determined that the switch sticker corresponding to the switch sticker image is pressed by a finger.

[0139] In one embodiment, the central control module 502 is used to receive at least one switch sticker image from the vision module. The switch sticker image is obtained by multi-angle collection of the preset switch sticker by at least one camera in the vision module;

[0140] The central control module 502 is used to perform differential calculation, determine the confidence corresponding to each switch sticker image based on the image change situation in at least one switch sticker image, and determine whether the switch sticker is pressed by a finger based on at least one confidence.

[0141] It should be noted that: when the recognition system provided in the above embodiment realizes the vision recognition and switch sticker interaction method of the corresponding vehicle, only the above division of each program module is used for illustration. In actual application, the above processing can be allocated to different program modules according to needs, that is, the internal structure of the above system is divided into different program modules to complete all or part of the above description of the processing. In addition, the system provided in the above embodiment and the corresponding Figure 1bEmbodiments of the method described above belong to the same inventive concept. For the specific implementation process, please refer to the method embodiments and will not be elaborated here.

[0142] Embodiments of the present disclosure also provide a computer device having the above-mentioned Figure 5 shown recognition system.

[0143] Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of another recognition system provided by embodiments of the present disclosure. As Figure 6 shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some alternative embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a set of blade servers, or a multi-processor system). Figure 6 One processor 10 is taken as an example in

[0144] The processor 10 can be a central processing unit, a network processor, or a combination thereof. Among them, the processor 10 can further include a hardware chip. The above-mentioned hardware chip can be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The above-mentioned programmable logic device can be a complex programmable logic device, a field programmable gate array, a general array logic, or any combination thereof.

[0145] Among them, the memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiments.

[0146] The memory 20 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 can optionally include a memory remotely set relative to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an enterprise internal network, a local area network, a mobile communication network, and combinations thereof.

[0147] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, a hard disk, or a solid state drive; the memory 20 may further include a combination of the above types of memory.

[0148] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30, and the output device 40 may be connected by a bus or other means. Figure 6 Taking connection by a bus as an example.

[0149] The input device 30 can receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED), and a haptic feedback device (e.g., a vibration motor), etc. The above display device includes but is not limited to a liquid crystal display, a light emitting diode, a display, and a plasma display. In some alternative embodiments, the display device may be a touch screen.

[0150] The computer device further includes a communication interface for the computer device to communicate with other devices or a communication network.

[0151] The embodiments of the present disclosure further provide a computer-readable storage medium. The methods according to the embodiments of the present disclosure may be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented as computer code that is originally stored in a remote storage medium or a non-transitory machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the methods described herein can be stored in such software processes on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium may be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state drive, etc.; further, the storage medium may further include a combination of the above types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the methods shown in the above embodiments are implemented.

[0152] Part of the present disclosure can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the present disclosure through the operations of the computer. Those skilled in the art should understand that the forms of existence of computer program instructions in a computer-readable medium include but are not limited to source files, executable files, installation package files, etc. Correspondingly, the ways for a computer to execute computer program instructions include but are not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Herein, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible by the computer.

[0153] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for visual recognition of a vehicle and interaction with a switch sticker, characterized in that: Applied to a recognition system, the recognition system includes: a visual module, a central control module and an execution module, and the method includes: A visual module, using at least one camera to collect images of a preset switch sticker and determine an image of the switch sticker; The central control module uses an image algorithm to extract features from the switch sticker image, identifies whether the switch sticker is manually pressed in the switch sticker image, and sends a control instruction to the execution module when it is determined that the switch sticker is manually pressed; The execution module receives the control instruction from the central control module, and executes the preset operation corresponding to the switch sticker based on the control instruction.

2. The method according to claim 1, characterized in that The switch sticker includes at least one of the following: a physical sticker with a preset characteristic pattern or a characteristic point at a preset position of the vehicle.

3. The method according to claim 2, characterized in that The method further comprises: a visual module, when the first switch sticker is a physical sticker with a preset characteristic pattern, using at least one camera to collect an image of the first switch sticker after the first switch sticker is installed to determine an image of the first switch sticker; The central control module determines the first coordinates of the first switch sticker based on the first switch sticker image, and identifies the preset characteristic pattern in the first switch sticker image; The central control module associates the first switch sticker with a preset operation of the execution module based on the preset characteristic pattern, determines a first association relationship, and saves the preset characteristic pattern, the first coordinate, and the first association relationship; The visual module, when the second switch sticker is a feature point at a preset position of the vehicle, uses the at least one camera to collect an image of the feature point at the preset position of the vehicle to determine an image of the second switch sticker; The central control module determines the second coordinates of the second switch sticker based on the second switch sticker image, and identifies the feature point in the second switch sticker image; The central control module associates the second switch sticker with the preset operation of the execution module based on the feature point, determines a second association relationship, and saves the feature point, the second coordinates, and the second association relationship.

4. The method according to claim 3, characterized in that The central control module uses an image algorithm to extract features from the switch sticker image, and identifies whether the switch sticker is manually pressed in the switch sticker image, based on the following steps: The central control module determines whether the switch sticker image is blocked, and when the switch sticker image is blocked, performs image expansion processing on the switch sticker image to determine the switch sticker expanded image; The central control module determines whether the switch sticker expanded image is completely blocked based on the blocked area of ​​the preset characteristic pattern or characteristic point in the switch sticker expanded image; The central control module, when the expanded image of the switch sticker is not completely blocked, identifies whether there is a hand feature in the expanded image of the switch sticker, and when it is determined that the hand feature exists, determines that the switch sticker corresponding to the switch sticker image is pressed by a finger; The central control module, when the expanded image of the switch sticker is completely blocked, identifies whether there is a hand feature in the expanded image of the switch sticker, and when it is determined that the hand feature exists, determines that the switch sticker corresponding to the switch sticker image is pressed by a finger.

5. The method according to claim 4, characterized in that The method further comprises: The central control module receives at least one switch sticker image from the visual module, wherein the switch sticker image is acquired by collecting a preset switch sticker from multiple angles by at least one camera in the visual module; The central control module uses differential calculation to determine the confidence level corresponding to each switch sticker image based on the image change in the at least one switch sticker image, and determines whether the switch sticker is pressed by a finger based on at least one confidence level.

6. A recognition system, characterized in that: The recognition system includes a visual module, a central control module and an execution module, wherein: A visual module, used to collect images of a preset switch sticker using at least one camera to determine an image of the switch sticker; A central control module, configured to extract features from the switch sticker image using an image algorithm, identify whether the switch sticker is manually pressed in the switch sticker image, and send a control instruction to an execution module when it is determined that the switch sticker is manually pressed; An execution module is used to receive the control instruction from the central control module, and execute a preset operation corresponding to the switch sticker based on the control instruction.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the vehicle visual recognition and switch sticker interaction method according to any one of claims 1 to 5.

8. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the vehicle visual recognition and switch sticker interaction method according to any one of claims 1 to 5.

Images