Viewing and aiming equipment control method and system, program product and equipment

Abstract

The invention discloses an observing and aiming equipment control method and system, a program product and equipment. The method comprises the following steps: acquiring initial sensor data for indicating the moving action of observing and aiming equipment; based on the initial sensor data, forming sensor input data of a pre-trained action recognition model; inputting data based on the sensor, and outputting an action recognition result through the action recognition model; and controlling the observing and aiming equipment to execute a corresponding control instruction based on the action recognition result.

Description

Technical Field

[0001] The present invention relates to the field of equipment control technology, and in particular to an observation and aiming equipment control method, an observation and aiming equipment control system, a computer program product, and an observation and aiming equipment. Background Art

[0002] With the continuous updating and iteration of photoelectric sensor technology, the camera industry that collects light signals and converts them into image signals is constantly developing, and the image quality and other supporting functions provided by the camera are also constantly developing and enriching.

[0003] A camera is generally provided in the viewing and aiming device. In order to ensure the image quality during the use of the viewing and aiming device, the viewing and aiming device needs to be adjusted and controlled. In the existing technology, the user is generally required to manually adjust the parameters on the user interface of the viewing and aiming device to achieve control of the viewing and aiming device, and then continue to use the viewing and aiming device. The method and steps for controlling the viewing and aiming device are relatively cumbersome, which will affect the user experience during use. Summary of the Invention

[0004] In order to solve the existing technical problems, the present invention provides a sighting device control method, a sighting device control system, a computer program product and a sighting device, which can more flexibly control the sighting device and improve the user experience.

[0005] In a first aspect, a method for controlling an observation and aiming device is provided, comprising: acquiring initial sensor data indicating a movement action of the observation and aiming device; forming sensor input data of a pre-trained action recognition model based on the initial sensor data; outputting an action recognition result based on the sensor input data and through the action recognition model; and controlling the observation and aiming device to execute corresponding control instructions based on the action recognition result.

[0006] In a second aspect, an observation and aiming device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes an observation and aiming device control method as described in any one of the first aspects of the present application.

[0007] In a third aspect, a sighting device control system is provided, the system comprising a server and the sighting device described in any one of the second aspects of the present application, the server communicating with the sighting device.

[0008] In a fourth aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements a method for controlling an observation and aiming device as described in any one of the first aspects of the present application.

[0009] This application obtains initial sensor data when moving the aiming and viewing device, forms sensor input data of a pre-trained motion recognition model based on the initial sensor data, outputs motion recognition results through the motion recognition model, and controls the aiming and viewing device to perform corresponding actions based on the motion recognition results. This application forms the moving motion of the aiming and viewing device when the aiming and viewing device is moved, and the aiming and viewing device performs the corresponding action after recognizing the moving motion. Therefore, when using the aiming and viewing device, the user does not need to configure control parameters through the user interface, and can directly control the aiming and viewing device through the mobile aiming and viewing device, thereby more flexibly controlling the aiming and viewing device and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a diagram of an application environment of a method for controlling an observation and aiming device in one embodiment;

[0011] Figure 2 This is a flow chart of a method for controlling an observation and aiming device in one embodiment;

[0012] Figure 3 is a flow chart of a method for controlling an observation and aiming device in another embodiment;

[0013] Figure 4 is a schematic diagram of a sighting equipment control device in one embodiment;

[0014] Figure 5 Schematic diagram of an observation and aiming device in one embodiment. DETAILED DESCRIPTION

[0015] The technical solution of the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used herein in the specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the scope of protection of the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0017] In the following description, reference is made to “some embodiments” which describe a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0018] See Figure 1, is a diagram of the application environment of a method for controlling an observation device in one embodiment. The method for controlling an observation device is applied to an observation device 10, which includes an image acquisition device 12, a processor 13, a memory 14, and a motion sensor 15. The image acquisition device 12 is used to capture images of the scene in which the observation device 10 is located. The motion sensor 15 collects sensor data when the observation device 10 is moved. The processor 13 identifies the target movement based on the collected sensor data and controls the observation device 10 to execute the corresponding control instructions based on the target control operation matched to the target movement. The processor 13 processes and optimizes the images captured by the image acquisition device 12 on the monitored scene and displays the processed images. The memory 14 is used to store programs and various types of data corresponding to the implementation of the method for controlling the observation device. The observation device 10 can also communicate with the server 20 to upload the collected training sample dataset to the server 20. The server 20 trains the motion recognition model based on the training sample dataset and then sends the parameter file of the trained motion recognition model to the observation device 10. The parameter file can be stored in the memory 14 before the observation device 10 leaves the factory, or the parameter file can be requested from the server 20 during use. In other embodiments, it is understood that the server 20 may not be included, and the trained motion recognition model can be trained on other computing devices and stored in the memory before the observation device 10 leaves the factory.

[0019] The sighting and viewing device 10 is an electronic device with sighting and viewing functions, including but not limited to a telescope, a rifle scope, a hunting rifle, a vehicle-mounted device, a flight device, and the like.

[0020] The image acquisition device 12 may be a combination of one or more sensors. The image acquisition device 12 may be a monocular vision sensor or a multi-vision vision sensor. For example, the image acquisition device 12 may be a combination of one or more sensors selected from the group consisting of a thermal imaging sensor, a visible light image sensor, a millimeter wave sensor, a lidar sensor, an infrared thermal imaging sensor, and a depth sensor.

[0021] The processor 13 may be one or more processors. When there are multiple processors 13, the multiple processors may be integrated on one chip or independently set on each chip.

[0022] The motion sensor 15 includes but is not limited to one or more of the following: inertial measurement unit (IMU), velocity sensor, acceleration sensor, gyroscope sensor, geomagnetic sensor, rotation vector sensor, steering wheel angle sensor, level sensor, tilt sensor, vibration sensor, displacement sensor and gravity sensor, etc.

[0023] The sighting device 10 may also include other sensor modules, including but not limited to environmental perception sensors. Environmental perception sensors include but are not limited to one or more combinations of the following sensors: brightness sensor, temperature sensor, haze sensor, and other environmental sensors.

[0024] The viewing device 10 may further include a display terminal for displaying images.

[0025] With the continuous advancement of infrared photoelectric sensor technology, the infrared camera industry, which collects infrared rays and converts them into image signals, is constantly developing. The image quality and other supporting functions provided by infrared cameras are also constantly developing and enriching. To maintain image instructions, current infrared cameras require periodic shutter operation to optimize the image. Current shutter operation methods include directive or periodic. The underlying implementation logic of the aforementioned control strategy is still a fixed program running within the processor itself, or it waits to receive control instructions from the upper-level application chip to control the infrared camera. For infrared cameras as a whole, most control methods are based on communication protocol control using electrical signals. The existing technology does not implement control of the execution actions of the observation and aiming device based on the movement of the observation and aiming device itself in space.

[0026] See also Figure 2 , is a flow chart of a method for controlling an observation and aiming device provided in an embodiment of the present application. The method for controlling an observation and aiming device is applied to an observation and aiming device, and the method for controlling an observation and aiming device includes the following steps:

[0027] S10. Acquire initial sensor data indicating a movement action of the observation and aiming device.

[0028] In this embodiment, the moving action refers to the movement action of the viewing device when it is moved, and the moving action includes but is not limited to a combination of one or more of the following actions: linear motion in space, curved motion in space, wherein space includes any of the following: above, below, left, and right relative to the viewing device. The shape formed by the moving action can be regular or irregular, or a combination of the two. For example, when a user holds the viewing device and swings the viewing device, the shape of the moving action formed is a V-shape. When the viewing device is moved, the motion sensor can collect sensor data of the viewing device when it is moved. The collected sensor data can represent the motion data of the moving action, thereby facilitating the subsequent identification of the moving action based on the collected sensor data. The initial sensor data includes but is not limited to a combination of one or more of the following: acceleration data, velocity data, angular velocity data, moving direction, displacement, etc.

[0029] Each movement action corresponds to a control operation, which instructs the sighting device to execute an action. A control operation can be any of the following: switching the sighting device's color palette, closing or opening the shutter, switching algorithm parameters, increasing or decreasing the magnification, deactivating the aiming pattern, freezing the image, or releasing the image. Different movement actions correspond to different control operations, so different movement actions can be pre-set to cause the sighting device to execute different control instructions.

[0030] In an optional implementation, when the sighting device is in recognition mode and receives a recognition trigger instruction, it controls the motion sensor to collect multiple sets of initial sensor data. It then determines whether the multiple sets of initial sensor data meet a preset number. If the multiple sets of initial sensor data do not meet the preset number, it continues to collect initial sensor data. For example, one set of initial sensor data includes primary data collected by the accelerometer and primary data collected by the gyroscope. At least 150 sets of raw data from the accelerometer and gyroscope are collected, with each set of raw sensor data collected at an interval of approximately 10 milliseconds. Once the initial sensor data is ready, the data collected by the motion sensor can be read and the initial sensor data is recorded sequentially. The recognition trigger instruction can be a voice trigger instruction. For example, a user inputs voice information through a voice interface. When the sighting device recognizes that the voice information matches the voice used to control the sighting device in the recognition model, the recognition trigger instruction is obtained. The recognition trigger instruction can also be a command obtained through the user interface or a command matched by a preset movement action.

[0031] S11. Based on the initial sensor data, sensor input data of a pre-trained motion recognition model is formed.

[0032] In this embodiment, the motion recognition model is a machine learning model trained based on a training sample dataset. Each training sample in the training sample dataset includes sensor sample data and a motion label corresponding to the sensor sample data. The motion label is the action indicated by the sensor sample data. For example, the motion label can be used to repeatedly swing the sighting device so that it moves in an up-and-down straight line, and then repeatedly collect sensor sample data for this up-and-down straight line movement. Using the training sample dataset, the motion recognition model can learn the characteristics of sensor data under various motions, that is, learn the spatial motion characteristics of the sighting device, and thus, after training, identify the motion of the sighting device. For example, by repeatedly collecting the data characteristics of a specific motion, a parameter file is generated after training. The parameter file can be used to generate new firmware code, which can be used to identify the specific motion and thus control the shutter.

[0033] In this embodiment, since the initial sensor data is collected raw and may contain some noise, it is necessary to preprocess the initial sensor data to obtain sensor input data for the action recognition model. This preprocessing includes, but is not limited to, eliminating data that is not within a preset expected range, eliminating noise points, and so on.

[0034] S12. Based on the sensor input data, the action recognition result is output through the action recognition model.

[0035] In this embodiment, the action recognition model extracts spatial motion feature data based on sensor input data through multiple convolutions, then performs maximum pooling on the spatial motion feature data through a pooling layer, processes it through an activation layer, and finally obtains the action recognition result. The action recognition model outputs the action recognition result by calculating the similarity between the extracted spatial motion feature data and the feature data stored in the parameter file. The action recognition result includes but is not limited to the target movement action and the confidence level corresponding to the target movement action. The target movement action is the movement action identified by the action recognition model. The confidence level corresponding to the target movement action represents the recognition probability corresponding to the target movement action. The higher the confidence level, the higher the accuracy of the target movement action recognition; the lower the confidence level, the lower the accuracy of the target movement action recognition.

[0036] S13. Based on the action recognition result, control the sighting device to execute corresponding control instructions.

[0037] In this embodiment, the corresponding control command can be any of the following: switching the color palette of the viewing device, closing the shutter, opening the shutter, switching algorithm parameters, increasing or decreasing the magnification, disabling the aiming pattern, freezing the image, or releasing the image. For example, a leftward swing of the viewing device can be preconfigured to correspond to opening the shutter, while a rightward swing corresponds to closing the shutter. When the viewing device is swung left, a large amount of sensor sample data corresponding to the leftward swing is collected, and the motion label is "swing left." When the viewing device is swung right, a large amount of sensor sample data corresponding to the rightward swing is collected, and the motion label is "swing right." This generates a training sample dataset for training the motion recognition model. After training, when the viewing device is used, the shutter automatically opens when the leftward swing is detected, and automatically closes when the rightward swing is detected. This allows for flexible control of the viewing device without requiring additional parameter adjustments on the user interface.

[0038] In the above embodiment, initial sensor data is obtained when the viewing and aiming device is moved, sensor input data of a pre-trained motion recognition model is formed based on the initial sensor data, the motion recognition result is output through the motion recognition model, and based on the motion recognition result, the viewing and aiming device is controlled to perform a corresponding action. In the present application, a moving motion of the viewing and aiming device is formed when the viewing and aiming device is moved, and the viewing and aiming device performs a corresponding action after recognizing the moving motion. Therefore, when using the viewing and aiming device, the user does not need to configure control parameters through the user interface, and the viewing and aiming device can be controlled directly through the mobile viewing and aiming device, thereby more flexibly controlling the viewing and aiming device and improving the user experience.

[0039] In some embodiments, controlling the sighting device to execute a corresponding control instruction based on the action recognition result includes:

[0040] Acquire a target movement action and a confidence level of the target movement action according to the action recognition result;

[0041] When the confidence level is greater than a preset confidence level, obtaining a target control operation corresponding to the target movement action;

[0042] According to the target control operation, the sighting device is controlled to execute a control instruction corresponding to the target control operation.

[0043] In this embodiment, when the confidence level is greater than a preset confidence level, it indicates that the movement motion corresponding to the action that the user desires to control the viewing device to perform is successfully recognized. That is, when the user moves the viewing device, the movement motion of the viewing device is successfully recognized, such as successfully recognizing a leftward swing. When the confidence level is less than or equal to the preset confidence level, it indicates that the movement motion corresponding to the action that the user desires to control the viewing device to perform is not successfully recognized. In an optional implementation, if the movement motion corresponding to the action that the user desires to control the viewing device to perform is not successfully recognized, information indicating the movement of the viewing device can be displayed on the user interface. This information can be presented in the form of an image or video, so that the user can more accurately perform the movement of the viewing device based on the displayed information.

[0044] In the above embodiment, the confidence level during recognition is combined to determine whether the movement action is successfully recognized. Only when the movement action of the mobile sighting device is successfully recognized, the control instruction corresponding to the target control operation corresponding to the target movement action is executed, thereby improving the control accuracy of the sighting device.

[0045] In some embodiments, the control instruction corresponding to the target control operation is any one of the following: switching the color palette of the sighting device, closing the shutter, opening the shutter, switching algorithm parameters, increasing the magnification, decreasing the magnification, turning off the aiming core pattern, freezing the picture, and releasing the picture.

[0046] In this embodiment, algorithm parameters are one or more parameters used to execute the aiming device control method. The aiming pattern can be any regular or irregular geometric shape, such as a cross. Freezing an image involves capturing an image and displaying it for a fixed period of time. Releasing an image removes the fixed display for a set period of time, effectively reversing the freeze operation. For any of the aforementioned target control operations, corresponding movement actions can be preconfigured, enabling control of the aiming device through the mobile aiming device.

[0047] In the above embodiment, the viewing and aiming device is moved to form a moving action of the viewing and aiming device. After the viewing and aiming device recognizes the moving action, it executes the corresponding action. Therefore, when using the viewing and aiming device, the user does not need to configure the control parameters through the user interface. The viewing and aiming device can be controlled directly by moving the viewing and aiming device, thereby more flexibly controlling the viewing and aiming device and improving the user experience.

[0048] In some embodiments, the sensor input data for forming the action recognition model based on the initial sensor data includes:

[0049] Acquiring valid sensor data within a preset expected range from the initial sensor data;

[0050] The mean value of the valid sensor data is obtained, and the absolute value of the difference between each valid sensor data and the mean value is calculated, and the valid sensor data with the absolute value of the difference greater than a preset threshold are removed to obtain the sensor input data.

[0051] In this embodiment, valid sensor data includes, but is not limited to, valid acceleration data, valid velocity data, valid angular velocity data, valid movement direction, valid displacement, and the like. For each type of sensor data, a preset expected range may be associated with it. Different types of sensor data may have different corresponding preset expected ranges.

[0052] For any type of initial sensor data, the above steps can be performed separately. For example, for acceleration data, a data filter can be used to eliminate abnormal values: first, a search is performed based on the set abnormal value to remove those acceleration data that are not within the preset expected range to obtain valid acceleration data, such as excessive acceleration that is unlikely to occur in the equipment working condition. Then, all valid acceleration data are averaged to obtain the acceleration average value. Then, another search is performed, and the absolute value of the difference between each valid acceleration speed and the acceleration average value is calculated to obtain the acceleration absolute difference. The values whose acceleration absolute difference is greater than the preset threshold set by the filter limit are removed again to obtain the acceleration input data.

[0053] In the above embodiment, the collected initial sensor data is processed to remove abnormal values to obtain more accurate sensor input data, thereby reducing the interference of noise data on subsequent action recognition and improving the accuracy of the control of the observation and aiming equipment.

[0054] In some embodiments, outputting the action recognition result based on the sensor input data and using the action recognition model includes:

[0055] Extracting spatial motion feature data using a feature extraction network in the motion recognition model based on the sensor input data;

[0056] Based on the spatial motion feature data, the action recognition network in the action recognition model is used to output the action recognition result.

[0057] In this embodiment, the action recognition model includes a feature extraction network and an action recognition network, wherein the feature extraction network includes a plurality of convolutional networks, and the action recognition network includes but is not limited to convolutional layers, pooling layers, activation layers, and the like. Feature data in the sensor input data is extracted multiple times through a plurality of convolutional networks to obtain spatial motion feature data, wherein the spatial motion feature data can exist in the form of a feature map. The pooling layer is used to reduce the spatial dimension (i.e., width and height) of the feature map while retaining important feature information. The activation layer enables the neural network to learn and simulate complex functional relationships by introducing nonlinear factors, while avoiding the problems of gradient disappearance and gradient explosion, thereby accelerating the training process.

[0058] In the above embodiment, spatial motion feature data representing the moving motion of the mobile observation and aiming device is extracted from the sensor input data through a pre-trained motion recognition model. Based on the spatial motion feature data, the motion recognition network in the motion recognition model is used to output the motion recognition result, which can improve the accuracy of motion recognition and thus improve the accuracy of the control of the observation and aiming device.

[0059] In some embodiments, the method further comprises:

[0060] Obtaining input information indicating user-selected movement actions and control operations;

[0061] Based on the input information, a selected movement action and a selected control operation are acquired, and the selected movement action and the selected control operation are bound.

[0062] In this embodiment, the input information indicates user input for binding a movement action to a control operation. The input information may be voice information or trigger operation information acquired through a user interface. For example, a user may input, "When the movement action of the mobile viewing device is a leftward swing, the corresponding control operation is to open the shutter," through a voice interface. Semantic analysis and word feature extraction are performed on the acquired voice input information to obtain movement action keywords and control operation keywords. The movement action corresponding to the movement action keyword is then bound to the control operation corresponding to the control operation keyword.

[0063] Optionally, the acquiring of input information for indicating the user's selection of a movement action and a control operation includes:

[0064] Based on the user interface, a triggering operation is obtained; and according to the triggering operation, a selection operation of a triggered mobile action control and a selection operation of a triggered control operation control are obtained;

[0065] The acquiring of the selected movement action and the selected control operation based on the input information further includes:

[0066] A selected movement action is acquired based on the selected operation of the movement action control, and a selected control operation is acquired based on the selected operation of the control operation control.

[0067] In this embodiment, the user interface provides a movement action control and a control operation control. The movement action control allows you to select the desired movement action, and the control operation control allows you to select the desired control operation. The control can be in the form of a text box, button, drop-down box, etc. The user interface can also display bound movement actions and control operations, making it easier for users to intuitively understand the bound movement actions and control operations. After the user triggers the movement action control, the selected operation of the movement action control is obtained, and the movement action corresponding to the selected operation of the movement action control is obtained. After the user triggers the control operation control, the selected operation of the control operation control is obtained, and the control operation corresponding to the selected operation of the control operation control is obtained.

[0068] In an optional method, the control operations of the viewing device are displayed in categories on the user interface. For example, the shutter-related control operations are displayed in the first partition of the user interface, and the algorithm parameter-related control operations are displayed in the second partition. Through the classified display, it is convenient for the user to intuitively configure the movement actions corresponding to each control operation, so that the user does not need to manually search for the required functions.

[0069] In the above embodiment, by customizing the correspondence between movement actions and control operations, the control of the viewing device is made more in line with the user's habits, thereby improving the accuracy of the control of the viewing device and improving the user's experience of using the viewing device.

[0070] In some embodiments, the method further comprises:

[0071] A training sample data set is obtained, where each training sample in the training sample data set includes sensor sample data and a motion action label corresponding to the sensor sample data.

[0072] Optionally, obtaining a training sample data set further includes:

[0073] Get the command for collecting data;

[0074] According to the data collection command, a group of sensor sample data is collected;

[0075] When it is determined that the sensor sample data does not meet the sample condition, continue to collect a new set of sensor sample data; when it is determined that the sensor sample data meets the sample condition, record the sensor sample data that meets the sample condition in sequence.

[0076] In this embodiment, the data collection command can be a user inputting data collection information through an input interface, such as a voice interface or user interface. The observation device then receives the data collection command based on the data collection information, thereby collecting training sample data. The data collection command can also be a command sent by a server in communication with the observation device. After receiving the data collection command, the observation device enters training data collection mode, then moves the observation device to form a preset movement motion, and then collects sensor sample data corresponding to the movement motion. Repeating this movement motion multiple times can generate a set of sensor sample data. The sample condition includes a preset number of sets. If the number of sensor sample data sets is less than the preset number, sensor sample data collection continues until the preset number of sets is met, i.e., the number of samples required for a single recognition session is met. For example, the collection interval between each set of sensor sample data is approximately 10 milliseconds. Once the sensor sample data is ready, the sensor sample data is recorded sequentially. When the recorded data set meets the number of data required for a single training session, the collection of the training sample data set is complete.

[0077] In the above embodiment, a training sample data set can be collected through the observation and aiming device to provide rich training samples for the subsequent training of the motion recognition model, thereby accelerating the training of the motion recognition model, improving the accuracy of mobile motion recognition, improving the accuracy of the control of the observation and aiming device, and improving the user experience of using the observation and aiming device.

[0078] In some embodiments, the present application further provides an observation and aiming device control system, the system comprising a server and the observation and aiming device described in any embodiment of the present application, the server communicating with the observation and aiming device.

[0079] In some embodiments, the observation device sends a training sample data set to the server, the server receives the training sample data set, constructs an initial motion recognition model, and iteratively trains the initial motion recognition model based on the training sample data set using an iterative method. After the iteration termination condition is met, a pre-trained motion recognition model is obtained.

[0080] In this embodiment, the observation and aiming device communicating with the server can be one or more observation and aiming devices. The step of training the motion recognition model is performed on the server. The server determines whether the amount of training data in the training sample dataset meets the preset training sample amount. If the amount of training data in the training sample dataset does not reach the preset training sample amount, the server sends a data collection command to the observation and aiming device. After receiving the data collection command, the observation and aiming device continues to collect sensor sample data. The training sample dataset may include sensor sample data corresponding to multiple motion labels. Sensor sample data corresponding to the same motion label are collected separately. During each iteration, the server obtains training samples from the training sample dataset for training, obtains the motion corresponding to the training sample during each iteration, calculates the loss value between the motion corresponding to the training sample and the motion label corresponding to the training sample during each iteration, and obtains the loss value during each iteration. Based on the loss value, the server determines whether the current iteration meets the iteration termination condition. If the current iteration does not meet the iteration termination condition, the server continues to obtain training samples for training. If the current iteration meets the iteration termination condition, the motion recognition model after the termination of the iteration is used as the pre-trained motion recognition model.

[0081] It is understandable that the server needs to preprocess the training samples in the training sample data set. The preprocessing steps are similar to the preprocessing steps in the above recognition process. Only after preprocessing can the input data of the action recognition model in the iterative process be obtained, which will not be repeated here.

[0082] In the above embodiment, a training sample data set is obtained through an observation device, and a motion recognition model is trained on a server based on the training sample data set. Training the motion recognition model through the server can improve the training speed.

[0083] In some embodiments, as Figure 3 As shown, Figure 3 This is a flow chart of a method for controlling an observation and aiming device in another embodiment, the steps comprising:

[0084] S31. Obtain data acquisition command.

[0085] S32. Collect sensor sample data.

[0086] S33. Determine whether the number of sensor sample data meets the sample condition. If the number of sensor sample data does not meet the sample condition, return to execute S32. If the number of sensor sample data meets the sample condition, execute S34.

[0087] S34: Record the collected sensor sample data and send the collected sensor sample data to the server.

[0088] S35. The server receives the sensor sample data sent multiple times by the observation and aiming device.

[0089] S36: The server determines whether the amount of training data of the sensor sample data in the received training sample data set meets the preset training sample amount. If the amount of training data does not reach the preset training sample amount, the server sends a data collection command to the observation and aiming device. If the amount of training data is greater than or equal to the preset training sample amount, the server executes S37.

[0090] S37. Based on the training sample data set, train the action recognition model, obtain a parameter file, and send the parameter file to the observation and aiming device.

[0091] S38. Obtain recognition trigger instruction.

[0092] S39: Acquire the collected initial sensor data.

[0093] S40: Determine whether the amount of the collected initial sensor data meets the preset amount. If the amount of the collected initial sensor data is greater than or equal to the preset amount, execute S41. If the amount of the collected initial sensor data is less than the preset amount, return to execute S39.

[0094] S41 . Based on the initial sensor data, sensor input data of a pre-trained motion recognition model is formed; and based on the sensor input data, a motion recognition result is output through the motion recognition model.

[0095] S42: Determine whether the confidence level of the target movement action is greater than a preset confidence level. If the confidence level of the target movement action is equal to or less than the preset confidence level, do not execute the target control operation corresponding to the target movement action. If the confidence level of the target movement action is greater than the preset confidence level, execute S43: Based on the target control operation, control the sighting device to execute a control instruction corresponding to the target control operation.

[0096] On the other hand, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the sighting device control method described in any embodiment of the present application.

[0097] Among them, in the computer program product, an optional implementation form of the program module architecture of the computer program that implements each step of the observation and aiming device control method can be an observation and aiming device control device.

[0098] See also Figure 4 An embodiment of the present application provides a control device for an observation and aiming device, comprising: an acquisition module 41 for acquiring initial sensor data indicating a movement action of the observation and aiming device; a preprocessing module 42 for generating sensor input data for a pre-trained action recognition model based on the initial sensor data; a recognition module 43 for outputting an action recognition result based on the sensor input data and through the action recognition model; and a control module 44 for controlling the observation and aiming device to execute a corresponding control instruction based on the action recognition result.

[0099] Optionally, the control module 44 is further configured to:

[0100] Acquire a target movement action and a confidence level of the target movement action according to the action recognition result;

[0101] When the confidence level is greater than a preset confidence level, obtaining a target control operation corresponding to the target movement action;

[0102] According to the target control operation, the sighting device is controlled to execute a control instruction corresponding to the target control operation.

[0103] Optionally, the corresponding control instruction is any one of the following: switching the color palette of the observation and aiming device, closing the shutter, opening the shutter, switching algorithm parameters, increasing the magnification, decreasing the magnification, closing the aiming core pattern, freezing the picture, and releasing the picture.

[0104] Optionally, the pre-processing module 42 is further configured to:

[0105] Acquiring valid sensor data within a preset expected range from the initial sensor data;

[0106] The mean value of the valid sensor data is obtained, and the absolute value of the difference between each valid sensor data and the mean value is calculated, and the valid sensor data with the absolute value of the difference greater than a preset threshold are removed to obtain the sensor input data.

[0107] Optionally, the identification module 43 is further configured to:

[0108] Extracting spatial motion feature data using a feature extraction network in the motion recognition model based on the sensor input data;

[0109] Based on the spatial motion feature data, the action recognition network in the action recognition model is used to output the action recognition result.

[0110] Optionally, the identification module 43 is further configured to:

[0111] Obtaining input information indicating user-selected movement actions and control operations;

[0112] Based on the input information, a selected movement action and a selected control operation are acquired, and the selected movement action and the selected control operation are bound.

[0113] Optionally, the identification module 43 is further configured to:

[0114] Based on the user interface, a triggering operation is obtained; and according to the triggering operation, a selection operation of a triggered mobile action control and a selection operation of a triggered control operation control are obtained;

[0115] The acquiring of the selected movement action and the selected control operation based on the input information further includes:

[0116] A selected movement action is acquired based on the selected operation of the movement action control, and a selected control operation is acquired based on the selected operation of the control operation control.

[0117] Optionally, the identification module 43 is further configured to:

[0118] A training sample data set is obtained, where each training sample in the training sample data set includes sensor sample data and a motion action label corresponding to the sensor sample data.

[0119] Optionally, the identification module 43 is further configured to:

[0120] Get the command for collecting data;

[0121] According to the data collection command, a group of sensor sample data is collected;

[0122] When it is determined that the sensor sample data does not meet the sample condition, continue to collect a new set of sensor sample data; when it is determined that the sensor sample data meets the sample condition, record the sensor sample data that meets the sample condition in sequence.

[0123] See also Figure 5 In another aspect of an embodiment of the present application, an observation and aiming device 10 is provided, comprising a processor 13 and a memory 14. The memory 14 stores a computer program. When the computer program is executed by the processor, the processor 13 executes the steps of a method for controlling an observation and aiming device provided in any of the above embodiments of the present application.

[0124] The processor 13 serves as the control center, connecting all components of the sighting device via various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 14 and accessing data stored in the memory 14, it performs various functions of the sighting device and processes data. Optionally, the processor 13 may include one or more processing cores; preferably, the processor 13 integrates an application processor and a modem processor. The application processor primarily handles the operating system, user interfaces, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor 13.

[0125] Memory 14 can be used to store software programs and modules. Processor 13 executes various functional applications and data processing by running the software programs and modules stored in memory 14. Memory 14 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as sound playback or image playback), while the data storage area may store data generated based on the use of a sighting device. Furthermore, memory 14 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 14 may also include a memory processor to provide processor 13 with access to memory 14.

[0126] On the other hand, an embodiment of the present application further provides a storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of a sighting device control method provided in any of the above embodiments of the present application.

[0127] In another aspect of an embodiment of the present application, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method for controlling an observation and aiming device as described in any embodiment of the present application is implemented.

[0128] Those skilled in the art will appreciate that all or part of the processes in the methods provided in the above embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0129] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. The scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for controlling an observation and aiming device, characterized in that: include: Acquiring initial sensor data indicative of movement of the sighting device; forming sensor input data for a pre-trained action recognition model based on the initial sensor data; Based on the sensor input data, the action recognition result is output through the action recognition model; Based on the action recognition result, the viewing device is controlled to execute corresponding control instructions.

2. The sighting device control method according to claim 1, wherein: The controlling the sighting device to execute a corresponding control instruction based on the action recognition result includes: Acquire a target movement action and a confidence level of the target movement action according to the action recognition result; When the confidence level is greater than a preset confidence level, obtaining a target control operation corresponding to the target movement action; According to the target control operation, the sighting device is controlled to execute a control instruction corresponding to the target control operation.

3. The sighting device control method according to claim 1, wherein: The corresponding control instruction is any one of the following: switching the color palette of the sighting device, closing the shutter, opening the shutter, switching algorithm parameters, increasing the magnification, decreasing the magnification, closing the aiming core pattern, freezing the picture, and releasing the picture.

4. The sighting device control method according to claim 1, wherein: The sensor input data for forming the action recognition model based on the initial sensor data includes: Acquiring valid sensor data within a preset expected range from the initial sensor data; The mean value of the valid sensor data is obtained, and the absolute value of the difference between each valid sensor data and the mean value is calculated, and the valid sensor data with the absolute value of the difference greater than a preset threshold are removed to obtain the sensor input data.

5. The sighting device control method according to claim 1, wherein: Outputting the action recognition result based on the sensor input data and through the action recognition model includes: Extracting spatial motion feature data using a feature extraction network in the motion recognition model based on the sensor input data; Based on the spatial motion feature data, the action recognition network in the action recognition model is used to output the action recognition result.

6. The sighting device control method according to claim 1, wherein: The method further comprises: Obtaining input information indicating user-selected movement actions and control operations; Based on the input information, a selected movement action and a selected control operation are acquired, and the selected movement action and the selected control operation are bound.

7. The sighting device control method according to claim 6, wherein: The obtaining of input information for indicating the user's selected movement action and control operation includes: Based on the user interface, a triggering operation is obtained; and according to the triggering operation, a selection operation of a triggered mobile action control and a selection operation of a triggered control operation control are obtained; The acquiring of the selected movement action and the selected control operation based on the input information further includes: A selected movement action is acquired based on the selected operation of the movement action control, and a selected control operation is acquired based on the selected operation of the control operation control.

8. The sighting device control method according to claim 1, wherein: The method further comprises: A training sample data set is obtained, where each training sample in the training sample data set includes sensor sample data and a motion action label corresponding to the sensor sample data.

9. The sighting device control method according to claim 8, wherein: The obtaining of the training sample data set further comprises: Get the command for collecting data; According to the data collection command, a group of sensor sample data is collected; When it is determined that the sensor sample data does not meet the sample condition, continue to collect a new set of sensor sample data; when it is determined that the sensor sample data meets the sample condition, record the sensor sample data that meets the sample condition in sequence.

10. A sighting device, characterized in that: The device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes a sighting device control method according to any one of claims 1 to 9.

11. A sighting equipment control system, characterized in that: The system includes a server and the observation and aiming device according to claim 10, wherein the server communicates with the observation and aiming device.

12. The sighting equipment control system according to claim 11, characterized in that: The system further comprises: The observation device sends a training sample data set to the server. The server receives the training sample data set, constructs an initial action recognition model, and iteratively trains the initial action recognition model based on the training sample data set using an iterative method. After the iteration termination condition is met, a pre-trained action recognition model is obtained.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for controlling an observation and aiming device as claimed in any one of claims 1 to 9 is implemented.

Images