A module identification method and related products

Abstract

This application provides a module recognition method and related products. The method includes: acquiring pattern acquisition data from an image acquisition unit; determining a target image analysis strategy for the pattern acquisition data based on the hardware type of the image acquisition unit and the pattern type of a preset pattern; determining the state information of the soft joystick module based on the pattern acquisition data and the target image analysis strategy; and transmitting the state information to a host computer through a communication interface. By analyzing the image acquisition data, the state information of the soft joystick module is obtained without physical contact with physical components or additional sensors, which helps reduce costs and improves the accuracy and intelligence of the state information recognition of the soft joystick module.

Description

Technical Field

[0001] This application belongs to the field of joystick module recognition technology, specifically relating to a module recognition method and related products. Background Technology

[0002] Currently, there are two main joystick recognition solutions: displacement detection and touch recognition. Displacement detection solutions include two types of joysticks: carbon film / potentiometer joysticks and Hall effect / tunnel magnetoresistive joysticks. Carbon film / potentiometer joysticks suffer from physical wear; after prolonged use, carbon powder shedding can lead to poor contact and joystick drift. Hall effect / tunnel magnetoresistive joysticks are susceptible to interference from magnets or surrounding circuitry, are more expensive, and consume more power. Touch recognition solutions suffer from fatigue-induced breakage of the flying wires, causing touch function failure. The wires can interfere with the casing, affecting the feel. Wear of the conductive coating reduces sensitivity. Furthermore, neither solution can detect the joystick's rotation or deformation around its axis, resulting in poor interactive functionality. Summary of the Invention

[0003] This application provides a module identification method and related products. By analyzing image acquisition data, the state information of the soft joystick module can be obtained. This method does not require physical contact with physical components or additional sensors, which helps reduce costs and improves the accuracy and intelligence of the state information identification of the soft joystick module.

[0004] In a first aspect, embodiments of this application provide a module identification method applied to a processing unit in a module identification system. The module identification system includes a soft joystick module, an image acquisition unit, the processing unit, and a communication interface. The image acquisition unit is used to acquire a preset pattern on the soft joystick module to obtain pattern acquisition data. The method includes: Acquire pattern acquisition data from the image acquisition unit; Based on the hardware type of the image acquisition unit and the pattern type of the preset pattern, determine the target image analysis strategy for the pattern acquisition data; Based on the pattern acquisition data and target image analysis strategy, the state information of the soft joystick module is determined; The status information is transmitted to the host through the communication interface.

[0005] In one possible example, determining the target image analysis strategy for the pattern acquisition data based on the hardware type of the image acquisition unit and the pattern type of the preset pattern includes: When the hardware type of the image acquisition unit is an event camera, the target image analysis strategy for the pattern acquisition data is determined to be the first image analysis strategy. When the hardware type of the image acquisition unit is a camera and the pattern type is a target customized pattern, the target image analysis strategy for the pattern acquisition data is determined to be the second image analysis strategy.

[0006] In one possible example, determining the state information of the soft joystick module based on the pattern acquisition data and the target image analysis strategy includes: When the target image analysis strategy is the second image analysis strategy, the first displacement information and the first deformation information of the continuous inter-frame texture are determined according to the pattern acquisition data and the preset optical flow algorithm; or, the first displacement information and the first deformation information of the continuous inter-frame texture are determined according to the pattern acquisition data and the first preset neural network model. The state information of the soft joystick module is determined based on the first displacement information and the first deformation information.

[0007] In one possible example, the state information of the soft joystick module is determined based on the second displacement information and the third deformation information, including: Based on the first displacement information and the first deformation information, determine the translation component, compression component, tension component and rotation transformation component; Based on the translation component, the movement operation information of the soft joystick module is determined; Based on the compression component, the pressing operation information of the soft joystick module is determined; Based on the stretching component, the extrusion operation information of the soft joystick module is determined; The rotation operation information of the soft joystick module is determined based on the rotation transformation component.

[0008] In one possible example, after determining the state information of the soft joystick module based on the first displacement information and the first deformation information, the method further includes: Fingerprint recognition is performed based on the pattern acquisition data to determine user information; Based on the user information, determine the output adjustment strategy of the software joystick module; The output curve of the software joystick module is controlled and adjusted according to the output adjustment strategy.

[0009] In one possible example, after determining the state information of the soft joystick module based on the first displacement information and the first deformation information, the method further includes: Based on the data collected from the pattern, the type of object in contact is determined; Determine the target texture identifier based on the type of the contacting object; Based on the target texture identifier, determine the target interaction command.

[0010] In one possible example, determining the state information of the soft joystick module based on the pattern acquisition data and the target image analysis strategy includes: Preprocess the pattern acquisition data; Based on the pattern acquisition data, determine the model input data; The model input data is input into a second preset neural network model to obtain the second deformation information and the second displacement information of the soft joystick module, and the second deformation information and the second displacement information are used as the state information of the soft joystick module.

[0011] In one possible example, determining the model input data based on the pattern acquisition data includes: Motion features are extracted from the preprocessed pattern acquisition data, and these motion features are used as input data for the model; or, Temporal features of the preprocessed pattern acquisition data are extracted using a temporal convolutional network, and these features are then used as input data for the model; or... The preprocessed pattern acquisition data is restored to the target image using an event and image transformation network, and the target image is used as the model input data.

[0012] Secondly, embodiments of this application provide a module identification system, the module identification system comprising: A soft joystick module, comprising a deformation component, an optical path component, an illumination component, an imaging component, a housing, a press button, a fixing component and a limiting component for the deformation component, wherein the lower surface of the deformation component is provided with a preset pattern; An image acquisition unit is located directly below the software joystick module, and the image acquisition unit is used to acquire pattern acquisition data of a preset pattern. The processing unit is configured to receive pattern acquisition data from the image acquisition unit and determine the state information of the software joystick module based on the pattern acquisition data. A communication interface is provided, and the processing unit is used to transmit the status information to the host through the communication interface.

[0013] Thirdly, embodiments of this application provide a module identification device applied to a processing unit in a module identification system. The module identification system includes a soft joystick module, an image acquisition unit, the processing unit, and a communication interface. The image acquisition unit is used to acquire a preset pattern on the soft joystick module to obtain pattern acquisition data. The module identification device includes an acquisition unit, a determination unit, and a transmission unit. The acquisition unit is used to acquire pattern acquisition data from the image acquisition unit; The determining unit is used to determine the target image analysis strategy for the pattern acquisition data based on the hardware type of the image acquisition unit and the pattern type of the preset pattern. The determining unit is also used to determine the state information of the software joystick module based on the pattern acquisition data and the target image analysis strategy; The transmission unit is used to transmit the status information to the host through the communication interface.

[0014] A fourth aspect of this application provides an electronic device including: a processor and a memory; and one or more programs stored in the memory and configured to be executed by the processor, the programs including instructions for some or all of the steps as described in the first aspect.

[0015] A fifth aspect of this application provides a computer-readable storage medium for storing a computer program that causes a computer to perform some or all of the steps described in the first aspect of this application.

[0016] A sixth aspect of this application provides a computer program product, comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of this application. The computer program product may be a software installation package.

[0017] As can be seen, in this embodiment, pattern acquisition data from the image acquisition unit is first obtained. Then, based on the hardware type of the image acquisition unit and the pattern type of the preset pattern, a target image analysis strategy for the pattern acquisition data is determined. Next, based on the pattern acquisition data and the target image analysis strategy, the state information of the soft joystick module is determined. Finally, the state information is transmitted to the host through the communication interface. By analyzing the image acquisition data, the state information of the soft joystick module is obtained without physical contact with physical components or additional sensors, which helps reduce costs and improves the accuracy and intelligence of the state information recognition of the soft joystick module. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1This is a schematic flowchart of a module identification method provided in an embodiment of this application; Figure 2 This is a schematic diagram of a customized pattern provided in an embodiment of this application; Figure 3 This is a flowchart illustrating a method for determining status information provided in an embodiment of this application; Figure 4 This is a schematic diagram of another process for determining status information provided in an embodiment of this application; Figure 5 This is a schematic diagram of a preset pattern image provided in an embodiment of this application; Figure 6 This is a schematic diagram of another preset pattern image provided in an embodiment of this application; Figure 7 This is a schematic diagram of yet another preset pattern image provided in the embodiments of this application; Figure 8 This is a schematic diagram of a preset pattern portion image provided in an embodiment of this application; Figure 9 This is a schematic diagram of the architecture of a module identification system provided in an embodiment of this application; Figure 10 This is a block diagram of the functional units of a module identification device provided in an embodiment of this application. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0021] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] In the embodiments of this application, "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent the following three situations: A exists alone; A and B exist simultaneously; B exists alone. Among them, A and B can be singular or plural.

[0024] In this embodiment, the symbol " / " can indicate that the preceding and following objects are in an "or" relationship. Alternatively, the symbol " / " can also represent a division sign, i.e., performing a division operation. For example, A / B can mean A divided by B.

[0025] In the embodiments of this application, "at least one item" or its similar expression refers to any combination of these items, including any combination of a single item or a plurality of items. "One or more" means one or more, while "multiple" means two or more. For example, "at least one item" of a, b, or c can represent the following seven cases: a, b, c; a and b; a and c; b and c; a, b, and c. Each of a, b, and c can be an element or a set containing one or more elements.

[0026] In the embodiments of this application, "equal to" can be used with "greater than" and is applicable to technical solutions used when "greater than" is used; it can also be used with "less than" and is applicable to technical solutions used when "less than" is used. When "equal to" is used with "greater than", it is not used with "less than"; when "equal to" is used with "less than", it is not used with "greater than".

[0027] To better understand the solutions of the embodiments of this application, the electronic devices, related concepts and background that may be involved in the embodiments of this application will be introduced below.

[0028] The electronic device in this application embodiment is a device with wireless communication capabilities, and may be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal device, vehicle-mounted terminal device, industrial control terminal device, UE unit, UE station, mobile station, remote station, remote terminal device, mobile device, UE terminal device, wireless communication device, UE agent, or UE device, etc. The terminal device can be fixed or mobile. It should be noted that the terminal device can support at least one wireless communication technology, such as LTE, New Radio (NR), Wideband Code Division Multiple Access (WCDMA), etc. For example, terminal devices can be mobile phones, tablets, desktop computers, laptops, all-in-one computers, in-vehicle terminals, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, electronic devices or other processing devices connected to a wireless modem, wearable devices, terminal devices in future mobile communication networks, or terminal devices in future evolved public land mobile networks (PLMNs), etc.

[0029] Please see Figure 1 , Figure 1This is a flowchart illustrating a module identification method provided in an embodiment of this application. It is applied to a processing unit within a module identification system. The module identification system includes a soft joystick module, an image acquisition unit, the processing unit, and a communication interface. The image acquisition unit is used to acquire a preset pattern on the soft joystick module to obtain pattern acquisition data. Specifically, it includes the following steps: Step S101: Obtain pattern acquisition data from the image acquisition unit.

[0030] The soft joystick module includes a deformation component, which in turn includes an elastic body. The elastic body can be deformed by pressing, squeezing, or lateral pressure, allowing the soft joystick module to move in displacement and rotation. A preset pattern is set on the lower surface of the elastic body.

[0031] The image acquisition unit's hardware includes an event camera and a webcam. When a user presses, squeezes, or rotates the software joystick module, its surface deforms, causing changes in the position, shape, and brightness of the preset pattern. The image acquisition unit captures these changes at a high frame rate. The event camera only outputs pixel brightness change events. If the brightness change value exceeds a preset brightness threshold, the event camera outputs the pixels with the brightness change, forming a brightness change sequence, i.e., image acquisition data. The webcam acquires RGB or grayscale images of part or all of the pattern, i.e., image acquisition data.

[0032] The preset pattern can be printed using high-contrast printing or laser engraving. The size and spacing of the pattern are designed according to the camera resolution to ensure that the image acquisition unit can reliably recognize it after deformation. The preset pattern can be a custom pattern, a dot matrix pattern, etc., and is not limited here. For an example, please refer to [link to example]. Figure 2 , Figure 2 A custom pattern is shown.

[0033] Step S102: Determine the target image analysis strategy for the pattern acquisition data based on the hardware type of the image acquisition unit and the pattern type of the preset pattern.

[0034] Different hardware types and different preset patterns can be used to derive the status information of the software joystick module using different image analysis strategies.

[0035] Optionally, when the hardware type of the image acquisition unit is an event camera, the target image analysis strategy for the pattern acquisition data is determined to be a first image analysis strategy; when the hardware type of the image acquisition unit is a camera and the pattern type is a target customized pattern, the target image analysis strategy for the pattern acquisition data is determined to be a second image analysis strategy.

[0036] Since the image acquisition unit is an event camera, the first image analysis strategy processes the brightness change sequence to extract information such as the spatial position, displacement vector, and degree of deformation of feature points, and then calculates the joystick's X / Y axis displacement, Z-axis pressure, Yaw rotation angle, touch state, contact point coordinates, and squeeze gesture. Since the image acquisition unit is a camera, the second image analysis strategy processes RGB or grayscale images to extract information such as the spatial position, displacement vector, and degree of deformation of feature points, and then calculates the joystick's X / Y axis displacement, Z-axis pressure, Yaw rotation angle, touch state, contact point coordinates, and squeeze gesture.

[0037] Step S103: Determine the state information of the software joystick module based on the pattern acquisition data and target image analysis strategy.

[0038] The pattern acquisition data can be input into a trained neural network, which outputs the state information of the soft joystick module.

[0039] Please see Figure 5 and Figure 8 , Figure 5 It is a complete image of the preset pattern captured by the camera. Figure 8 The pattern portion is scraped out of the field of view, but the camera captures a partial image of the preset pattern. Based on this partial image, a target image analysis strategy can still be executed to determine the state information of the software joystick module.

[0040] Step S104: Transmit the status information to the host through the communication interface.

[0041] The host computer is the upper-level computer that receives and processes the data and performs monitoring and management.

[0042] As can be seen, in this embodiment, pattern acquisition data from the image acquisition unit is first obtained. Then, based on the hardware type of the image acquisition unit and the pattern type of the preset pattern, a target image analysis strategy for the pattern acquisition data is determined. Next, based on the pattern acquisition data and the target image analysis strategy, the state information of the soft joystick module is determined. Finally, the state information is transmitted to the host through the communication interface. By analyzing the image acquisition data, the state information of the soft joystick module is obtained without physical contact with physical components or additional sensors, which helps reduce costs and improves the accuracy and intelligence of the state information recognition of the soft joystick module.

[0043] Please see Figure 3 Regarding the determination of the state information of the soft joystick module based on the pattern acquisition data and target image analysis strategy, the above method may include the following steps: Step S301: When the target image analysis strategy is the second image analysis strategy, the first displacement information and the first deformation information of the continuous inter-frame texture are determined according to the pattern acquisition data and the preset optical flow algorithm; or, the first displacement information and the first deformation information of the continuous inter-frame texture are determined according to the pattern acquisition data and the preset neural network model.

[0044] The core assumption of the preset optical flow algorithm is that the pixel grayscale value of the same spatial point remains unchanged between adjacent frames, and only the spatial position shifts. For the same physical point in two consecutive frames, its grayscale value satisfies: I(x,y,t)=I(x+dx,y+dy,t+dt).

[0045] Where I(x,y,t) represents the gray level of pixel (x,y) at time t, dx and dy represent the spatial displacement of the pixel in the x and y directions, and dt represents the inter-frame time difference; by performing a Taylor expansion on the right side and ignoring higher-order terms, the basic constraint equation for optical flow can be derived: I x u+I y v+I t =0.

[0046] Among them, I x I y Let I be the gradient of the image in the x and y directions. t The gradient is the time dimension, and u and v are the instantaneous velocities of the pixel in the x and y directions, i.e., the optical flow vectors.

[0047] Since a single pixel cannot solve for two unknowns u and v with only one equation, the assumption of consistent motion within the local window is introduced, which assumes that all points in the neighborhood centered on the target pixel share the same optical flow.

[0048] By constructing a least-squares optimization problem, the optimal optical flow vector corresponding to this window can be obtained by solving it: min∑ W (I x u+I y v+I t ) 2 .

[0049] The optical flow field is calculated pixel by pixel, which is the set of all optimal optical flow vectors. The change of the optical flow vectors in the image space is calculated based on the optical flow field, and further texture displacement information and deformation information are obtained, namely the first displacement information and the first deformation information.

[0050] Among them, extracting texture features from image acquisition data can be achieved by using neural network models (e.g., MobileNet-like lightweight models, ViT global feature models, U-Net deformation field estimation models) to analyze the texture features of pattern acquisition data, thereby obtaining the first displacement information and the first deformation information of texture between consecutive frames.

[0051] Step S302: Determine the state information of the soft joystick module based on the first displacement information and the first deformation information.

[0052] As can be seen, in this example, optical flow algorithms or neural network models can be used to determine the first displacement information and the first deformation information of the texture between consecutive frames, and further determine the state information of the soft joystick module, which is beneficial to improving the intelligence and accuracy of the joystick module's state information recognition.

[0053] Please see Figure 4 Regarding determining the state information of the soft joystick module based on the second displacement information and the third deformation information, the above method may include the following steps: Step S401: Determine the translation component, compression component, tension component, and rotational transformation component based on the first displacement information and the first deformation information.

[0054] Among them, based on the spatial distribution of the preset pattern (i.e., the spatial arrangement of all textures in the initial state of the preset pattern) and geometric prior (including the shape and size of the preset pattern), the first displacement information and the first deformation information are mapped to the physical space of the soft joystick module, and the overall rigid motion component and the local flexible deformation component of the soft joystick module are separated. The overall rigid motion component includes translation component and rotation transformation component, and the local flexible deformation component includes compression component and stretching component.

[0055] Step S402: Determine the movement operation information of the soft joystick module based on the translation component.

[0056] Specifically, the movement operation information of the software joystick module is determined based on the direction and amplitude of the translation component, including the movement direction and movement amplitude.

[0057] Step S403: Determine the pressing operation information of the soft joystick module based on the compression component.

[0058] The pressing operation information of the soft joystick module, including pressing depth and pressing area, is determined based on the quantized value of the compression component, including deformation gradient and local compression amount.

[0059] Step S404: Determine the extrusion operation information of the soft rocker module based on the stretching component.

[0060] Specifically, based on the quantized values ​​of the stretching components, including the deformation gradient and local stretching amount, the extrusion operation information of the soft joystick module is determined, including the degree of extrusion and the force-bearing area.

[0061] Step S405: Determine the rotation operation information of the soft joystick module based on the rotation transformation component.

[0062] Specifically, the rotation operation information of the soft joystick module, including rotation direction and rotation amplitude, is determined based on the rotation angle and direction of the rotation transformation component.

[0063] As can be seen in this example, displacement and deformation information can be mapped to the physical space of the soft joystick module, and translation, compression, stretching and rotation components can be separated. Then, the operation information of the soft joystick module can be determined, which helps to improve the accuracy and intelligence of determining the state information of the soft joystick module.

[0064] In one possible example, after determining the state information of the soft joystick module based on the first displacement information and the first deformation information, the above method may include the following steps: performing fingerprint recognition based on the pattern acquisition data to determine user information; determining the output adjustment strategy of the soft joystick module based on the user information; and controlling and adjusting the output curve of the soft joystick module according to the output adjustment strategy.

[0065] Please refer to Figure 5 and Figure 6 , Figure 5 It is an image of a preset pattern captured by the camera in a static state, that is, without any object in contact. Figure 6 An image of a preset pattern captured by a camera when a person touches it with their finger.

[0066] Among them, the texture features, fingertip lines, skin folds, and pressure deformation distribution information of the pattern acquisition data can be extracted, and compared and verified with the pre-stored user fingerprint feature database to realize fingerprint identity recognition based on feature similarity.

[0067] When it is identified that a specific user is using the software joystick module, the user's operating habits can be known, and the output adjustment strategy of the software joystick module can be adjusted accordingly to match the user's operating feel.

[0068] As can be seen, in this example, fingerprint recognition of the user and corresponding adjustment of the output curve of the software joystick module can improve the user experience and enhance the intelligence of the software joystick module.

[0069] In one possible example, after determining the state information of the soft joystick module based on the first displacement information and the first deformation information, the above method may include the following steps: determining the contact object type based on the pattern acquisition data; determining the target texture identifier based on the contact object type; and determining the target interaction command based on the target texture identifier.

[0070] Please see Figure 7 , Figure 7 It is an image of a preset pattern captured by the camera after the gear contacts the soft joystick module. When the soft joystick module is used as a game joystick, it uses an object with a specific texture to contact the surface and outputs a specific texture ID (target texture identifier). In the game, the target interaction command can be executed based on this specific ID, such as unlocking corresponding functions or items in the game. For example, when the gear contacts the soft joystick module, the target item can be unlocked and purchased.

[0071] As can be seen in this example, different interactive experiences can be achieved by having objects with different specific textures touch the surface of the soft joystick module, which helps to improve the intelligence of the soft joystick module's interaction.

[0072] In one possible example, regarding the determination of the state information of the soft joystick module based on the pattern acquisition data and the target image analysis strategy, the above method may include the following steps: preprocessing the pattern acquisition data; determining model input data based on the pattern acquisition data; inputting the model input data into a second preset neural network model to obtain the second deformation information and the second displacement information of the soft joystick module; and using the second deformation information and the second displacement information as the state information of the soft joystick module.

[0073] Since the pattern acquisition data is a sequence of brightness changes, it can be preprocessed with noise filtering and event clustering.

[0074] Optionally, an optical flow algorithm can be used to extract motion features from the preprocessed pattern acquisition data and use the motion features as model input data; or, a temporal convolutional network (TCN) can be used to extract temporal features from the preprocessed pattern acquisition data and use the temporal features as model input data; or, an event and image transformation network can be used to restore the preprocessed pattern acquisition data into a target image and use the target image as model input data.

[0075] The second preset neural network model is the Vision Transformer (ViT) network model, which can calculate the second deformation information and the second displacement information of the soft joystick module in real time based on the model input data.

[0076] As can be seen, in this example, a neural network model can be used to process the brightness change sequence and calculate the second deformation information and second displacement information of the soft joystick module in real time, which is beneficial to improving the intelligence of determining the state of the soft joystick module.

[0077] Please see Figure 9 , Figure 9 This is a schematic diagram of the architecture of a module identification system provided in an embodiment of this application, such as... Figure 9 As shown, the module recognition system 1 includes a soft joystick module 10, an image acquisition unit 20, a processing unit 30, and a communication interface 40. The soft joystick module 10 includes a deformation component, an optical path component, an illumination component, an imaging component, a housing, a press button, a fixing component and a limiting component for the deformation component. A preset pattern is provided on the lower surface of the deformation component. The image acquisition unit 20 is located directly below the soft joystick module 10. The image acquisition unit 20 is used to acquire pattern acquisition data of the preset pattern. The processing unit 30 is used to receive the pattern acquisition data from the image acquisition unit 20 and determine the status information of the soft joystick module 10 based on the pattern acquisition data. The processing unit 30 is used to transmit the status information to the host 50 through the communication interface 40.

[0078] Specifically, the processing unit 30 can acquire pattern acquisition data from the image acquisition unit 20, determine the target image analysis strategy for the pattern acquisition data based on the hardware type of the image acquisition unit and the pattern type of the preset pattern, determine the status information of the software joystick module based on the pattern acquisition data and the target image analysis strategy, and transmit the status information to the host through the communication interface.

[0079] The deformation component includes an elastomer, a horizontally limiting cover for the elastomer, and a quick-release rotating base. The elastomer is axially limited and fixed by the limiting cover and the quick-release rotating base to achieve deformation under stress during human hand interaction.

[0080] The quick-release mounting base is fixed to the outer casing, and its inner wall is equipped with a guide rail that matches the quick-release rotating base. During assembly, the deformation component is inserted into the fixing structure axially and enters the locking position through a relative rotation at a preset angle. The locking component abuts against the positioning surface on the quick-release rotating base, achieving dual axial and radial limiting of the deformation system. During disassembly, the user presses a button to retract the locking flange, releasing the abutment relationship with the positioning surface, and then rotates it in the opposite direction to disengage the quick-release rotating base from the locked position, thereby unlocking and removing the deformation system module.

[0081] The optical path assembly includes a light-diffusing component and a light-shielding shell. After the deformation assembly is assembled, it, together with the limiting cover, the quick-release rotating base, and the circuit board, forms a closed optical path space to ensure the stability of the internal lighting environment. The lighting component can adopt multiple LED arrays or ring light strips, and optical simulation design is used to ensure uniform light reception on the pattern layer.

[0082] The illumination and imaging components are mounted together on a circuit board. The imaging component includes a camera, a camera base plate, a leveling screw, and a soft material shim. When the leveling screw is adjusted, the soft material shim is compressed and deformed, changing its support height and causing a change in the attitude of the camera base plate, thereby achieving fine adjustment of the camera's deflection angle.

[0083] As can be seen, in this embodiment of the application, the state information of the soft joystick module is obtained by analyzing the image acquisition data. This does not require physical contact with physical components or additional sensors, which helps to reduce costs and improve the accuracy and intelligence of the state information recognition of the soft joystick module.

[0084] When dividing each function into modules according to its corresponding function. Figure 10 This is a functional unit block diagram of a module identification device provided in an embodiment of this application, such as... Figure 10 As shown, the module identification device includes an acquisition unit 1001, a determination unit 1002, and a transmission unit 1003; wherein, The acquisition unit 1001 is used to acquire pattern acquisition data from the image acquisition unit; The determining unit 1002 is used to determine the target image analysis strategy of the pattern acquisition data based on the hardware type of the image acquisition unit and the pattern type of the preset pattern. The determining unit 1002 is further configured to determine the state information of the soft joystick module based on the pattern acquisition data and the target image analysis strategy; The transmission unit 1003 is used to transmit the status information to the host through the communication interface.

[0085] As can be seen from the embodiments of this application, the module recognition device can first acquire pattern acquisition data from the image acquisition unit, then determine the target image analysis strategy for the pattern acquisition data based on the hardware type of the image acquisition unit and the pattern type of the preset pattern, then determine the state information of the soft joystick module based on the pattern acquisition data and the target image analysis strategy, and finally transmit the state information to the host through the communication interface. By analyzing the image acquisition data, the state information of the soft joystick module can be obtained without physical contact with physical components or additional sensors, which helps to reduce costs and improve the accuracy and intelligence of the state information recognition of the soft joystick module.

[0086] In one possible example, regarding the determination of the target image analysis strategy for the pattern acquisition data based on the hardware type of the image acquisition unit and the pattern type of the preset pattern, the determining unit 1002 is specifically used for: When the hardware type of the image acquisition unit is an event camera, the target image analysis strategy for the pattern acquisition data is determined to be the first image analysis strategy. When the hardware type of the image acquisition unit is a camera and the pattern type is a target customized pattern, the target image analysis strategy for the pattern acquisition data is determined to be the second image analysis strategy.

[0087] In one possible example, regarding the determination of the state information of the soft joystick module based on the pattern acquisition data and target image analysis strategy, the determining unit 1002 is specifically used for: When the target image analysis strategy is the second image analysis strategy, the first displacement information and the first deformation information of the continuous inter-frame texture are determined according to the pattern acquisition data and the preset optical flow algorithm; or, the first displacement information and the first deformation information of the continuous inter-frame texture are determined according to the pattern acquisition data and the first preset neural network model. The state information of the soft joystick module is determined based on the first displacement information and the first deformation information.

[0088] In one possible example, regarding the determination of the state information of the soft joystick module based on the second displacement information and the third deformation information, the determining unit 1002 is specifically used for: Based on the first displacement information and the first deformation information, determine the translation component, compression component, tension component and rotation transformation component; Based on the translation component, the movement operation information of the soft joystick module is determined; Based on the compression component, the pressing operation information of the soft joystick module is determined; Based on the stretching component, the extrusion operation information of the soft joystick module is determined; The rotation operation information of the soft joystick module is determined based on the rotation transformation component.

[0089] In one possible example, after determining the state information of the soft joystick module based on the first displacement information and the first deformation information, the determining unit 1002 is further specifically used for: Fingerprint recognition is performed based on the pattern acquisition data to determine user information; Based on the user information, determine the output adjustment strategy of the software joystick module; The output curve of the software joystick module is controlled and adjusted according to the output adjustment strategy.

[0090] In one possible example, after determining the state information of the soft joystick module based on the first displacement information and the first deformation information, the determining unit 1002 is further specifically used for: Based on the data collected from the pattern, the type of object in contact is determined; Determine the target texture identifier based on the type of the contacting object; Based on the target texture identifier, determine the target interaction command.

[0091] In one possible example, regarding the determination of the state information of the soft joystick module based on the pattern acquisition data and target image analysis strategy, the determining unit 1002 is further specifically used for: Preprocess the pattern acquisition data; Based on the pattern acquisition data, determine the model input data; The model input data is input into a second preset neural network model to obtain the second deformation information and second displacement information of the soft joystick module, and the second deformation information and second displacement information are used as the state information of the soft joystick module. In a possible example, in determining the model input data based on the pattern acquisition data, the determining unit 1002 is further specifically used for: Motion features are extracted from the preprocessed pattern acquisition data, and these motion features are used as input data for the model; or, Temporal features of the preprocessed pattern acquisition data are extracted using a temporal convolutional network, and these features are then used as input data for the model; or... The preprocessed pattern acquisition data is restored to the target image using an event and image transformation network, and the target image is used as the model input data.

[0092] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0093] The electronic device provided in this embodiment is used to execute the above-described module identification method, and therefore can achieve the same effect as the above-described implementation method.

[0094] When using integrated units, the electronic device may include a processing module, a storage module, and a communication module. The processing module can be used to control and manage the actions of the electronic device; for example, it can support the electronic device in executing the steps performed by the aforementioned functional units. The storage module can support the electronic device in executing stored program code and data. The communication module can support communication between the electronic device and other devices.

[0095] The processing module can be a processor or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other electronic devices.

[0096] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0097] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer includes a control platform.

[0098] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0099] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0101] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0103] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0104] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0105] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A module identification method, characterized in that, A processing unit is applied in a module recognition system, the module recognition system including a soft joystick module, an image acquisition unit, the processing unit, and a communication interface. The image acquisition unit is used to acquire a preset pattern on the soft joystick module to obtain pattern acquisition data. The method includes: Acquire pattern acquisition data from the image acquisition unit; Based on the hardware type of the image acquisition unit and the pattern type of the preset pattern, determine the target image analysis strategy for the pattern acquisition data; Based on the pattern acquisition data and target image analysis strategy, the state information of the soft joystick module is determined; The status information is transmitted to the host through the communication interface.

2. The method according to claim 1, characterized in that, The step of determining the target image analysis strategy for the pattern acquisition data based on the hardware type of the image acquisition unit and the pattern type of the preset pattern includes: When the hardware type of the image acquisition unit is an event camera, the target image analysis strategy for the pattern acquisition data is determined to be the first image analysis strategy. When the hardware type of the image acquisition unit is a camera and the pattern type is a target customized pattern, the target image analysis strategy for the pattern acquisition data is determined to be the second image analysis strategy.

3. The method according to claim 2, characterized in that, The step of determining the state information of the software joystick module based on the pattern acquisition data and target image analysis strategy includes: When the target image analysis strategy is the second image analysis strategy, the first displacement information and the first deformation information of the continuous inter-frame texture are determined according to the pattern acquisition data and the preset optical flow algorithm; or, the first displacement information and the first deformation information of the continuous inter-frame texture are determined according to the pattern acquisition data and the first preset neural network model. The state information of the soft joystick module is determined based on the first displacement information and the first deformation information.

4. The method according to claim 3, characterized in that, Based on the second displacement information and the third deformation information, the state information of the soft joystick module is determined, including: Based on the first displacement information and the first deformation information, determine the translation component, compression component, tension component and rotation transformation component; Based on the translation component, the movement operation information of the soft joystick module is determined; Based on the compression component, the pressing operation information of the soft joystick module is determined; Based on the stretching component, the extrusion operation information of the soft joystick module is determined; The rotation operation information of the soft joystick module is determined based on the rotation transformation component.

5. The method according to claim 3, characterized in that, After determining the state information of the soft joystick module based on the first displacement information and the first deformation information, the method further includes: Fingerprint recognition is performed based on the pattern acquisition data to determine user information; Based on the user information, determine the output adjustment strategy of the software joystick module; The output curve of the software joystick module is controlled and adjusted according to the output adjustment strategy.

6. The method according to claim 3, characterized in that, After determining the state information of the soft joystick module based on the first displacement information and the first deformation information, the method further includes: Based on the data collected from the pattern, the type of object in contact is determined; Determine the target texture identifier based on the type of the contacting object; Based on the target texture identifier, determine the target interaction command.

7. The method according to claim 2, characterized in that, The step of determining the state information of the software joystick module based on the pattern acquisition data and target image analysis strategy includes: Preprocess the pattern acquisition data; Based on the pattern acquisition data, determine the model input data; The model input data is input into a second preset neural network model to obtain the second deformation information and the second displacement information of the soft joystick module, and the second deformation information and the second displacement information are used as the state information of the soft joystick module.

8. The method according to claim 3, characterized in that, The step of determining the model input data based on the pattern acquisition data includes: Motion features are extracted from the preprocessed pattern acquisition data, and these motion features are used as input data for the model; or, Temporal features of the preprocessed pattern acquisition data are extracted using a temporal convolutional network, and these features are then used as input data for the model; or... The preprocessed pattern acquisition data is restored to the target image using an event and image transformation network, and the target image is used as the model input data.

9. A module identification system, characterized in that, The module identification system includes: A soft joystick module, comprising a deformation component, an optical path component, an illumination component, an imaging component, a housing, a press button, a fixing component and a limiting component for the deformation component, wherein the lower surface of the deformation component is provided with a preset pattern; An image acquisition unit is located directly below the software joystick module, and the image acquisition unit is used to acquire pattern acquisition data of a preset pattern. The processing unit is configured to receive pattern acquisition data from the image acquisition unit and determine the state information of the software joystick module based on the pattern acquisition data. A communication interface is provided, and the processing unit is used to transmit the status information to the host through the communication interface.

10. A module identification device, characterized in that, A processing unit is used in a module recognition system. The module recognition system includes a soft joystick module, an image acquisition unit, the processing unit, and a communication interface. The image acquisition unit is used to acquire a preset pattern on the soft joystick module to obtain pattern acquisition data. The module recognition device includes an acquisition unit, a determination unit, and a transmission unit. The acquisition unit is used to acquire pattern acquisition data from the image acquisition unit; The determining unit is used to determine the target image analysis strategy for the pattern acquisition data based on the hardware type of the image acquisition unit and the pattern type of the preset pattern. The determining unit is also used to determine the state information of the software joystick module based on the pattern acquisition data and the target image analysis strategy; The transmission unit is used to transmit the status information to the host through the communication interface.

Images