Posture recognition method and device, electronic equipment and storage device

Abstract

The invention discloses a posture recognition method and device, electronic equipment and a storage device, and relates to the field of financial science and technology and the field of artificial intelligence. The method comprises the following steps: acquiring a target image; extracting at least one spatial feature corresponding to each target object in the target image through a first type of convolution branch of the neural network; and determining attitude information of each target object according to the at least one spatial feature corresponding to each target object in the target image. The technical problem that in the prior art, the object posture detection speed is low is solved.

Description

Technical Field

[0001] The present application relates to the fields of financial technology and security, and specifically, to a gesture recognition method, device, electronic device and storage device. Background Art

[0002] With the continuous growth of social needs and the continuous advancement of technology, intelligent video surveillance systems have gradually become an important tool for banks and other institutions to prevent, respond to emergencies, and make decisions. Traditional video surveillance only has a recording function and can only detect theft and property damage after they occur, which is not real-time; security personnel can only monitor the security of outlets in real time during work hours, but they cannot monitor all-weather and without blind spots, so traditional monitoring has limitations.

[0003] With the development of computational video technology, the application of intelligent video surveillance systems in scenarios such as banks has gradually increased. Through intelligent video analysis technology, it can detect and warn behaviors in video surveillance images in real time, thereby improving the security prevention capabilities of banks and other institutions. Behavior analysis is an important part of intelligent video surveillance systems, especially in scenarios such as banks, where the important role of behavior analysis is becoming more and more prominent. At present, the mainstream of behavior analysis technology is detection technology based on deep learning. However, the current deep learning detection technology still has the problems of large computational complexity and low detection accuracy, resulting in a very slow detection speed for object postures.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The main purpose of the present application is to provide a gesture recognition method, device, electronic device and storage device to at least solve the technical problem of slow detection speed of object gesture in the prior art.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of an embodiment of the present application, a posture recognition method is provided, including: acquiring a target image, wherein the target image includes at least one target object; extracting at least one spatial feature corresponding to each target object in the target image through a first type of convolution branch of a neural network; wherein the first type of convolution branch and the second type of convolution branch are connected in a parallel structure, and the number of channels of the second type of convolution branch is less than that of the first type of convolution branch, and the second type of convolution branch is used to perform feature fusion on at least one spatial feature corresponding to each target object extracted by the first type of convolution branch during the training stage of the neural network; and determining the posture information of each target object according to at least one spatial feature corresponding to each target object in the target image.

[0007] Optionally, the neural network includes: N feature processors connected in series, wherein N is an integer greater than 1, each of the N feature processors includes a first type of convolution branch, a second type of convolution branch, and an activation function, wherein the activation function in each feature processor is used to learn the features output by the convolution branch in the feature processor.

[0008] Optionally, the neural network includes N feature processors connected in series, including: an initial feature processor, wherein the initial feature processor is the first feature processor among the N feature processors, wherein the initial feature processor includes an initial convolution kernel composed of a first type of convolution branch and a second type of convolution branch, and the initial feature processor also includes an activation function connected in series with the initial convolution kernel.

[0009] Optionally, the N feature processors connected in series included in the neural network also include: N-1 target feature processors, wherein the target feature processor is a feature processor other than the initial feature processor among the N feature processors, the target feature processor includes a target convolution kernel composed of a first type of convolution branch, a second type of convolution branch and an identity branch, the target feature processor also includes an activation function connected in series with the target convolution kernel, and the identity branch of each target feature processor is used to input the output result of the feature processor of the previous layer corresponding to the target feature processor into the activation function of the target feature processor.

[0010] Optionally, the training process of the neural network includes the following steps: obtaining a training image and label information corresponding to the training image, wherein the training image includes at least one object, and the label information is used to annotate the actual posture information of each object in the training image; performing multiple iterative training on the neural network based on the training image and the label information, wherein the iterative training is used to update the weight parameters of the first type of convolution branch and the second type of convolution branch of the initial feature processor in the neural network, and / or, updating the weight parameters of the first type of convolution branch, the second type of convolution branch and the identity branch in the neural network.

[0011] Optionally, iterative training includes the following steps: inputting a training image into a neural network, processing the training image in sequence through N feature processors in the neural network; and updating the neural network based on the output result and label information of the last target feature processor in the neural network after processing the training image.

[0012] Optionally, in the process of processing the training image in sequence through N feature processors in the neural network, the posture recognition method also includes: detecting whether the identity branch and the first type of convolution branch in the i-th target feature processor have the same number of channels and step size, where i is a positive integer less than or equal to N-1; when it is detected that the identity branch and the first type of convolution branch in the i-th target feature processor have the same number of channels and step size, the output result of the feature processor of the previous level of the i-th target feature processor is input into the activation function in the i-th target feature processor through the identity branch in the i-th target feature processor; when it is detected that the identity branch and the first type of convolution branch in the i-th target feature processor do not have the same number of channels and step size, the output result of the feature processor of the previous level of the i-th target feature processor is input into the activation function in the i-th target feature processor through the first type of convolution branch and the second type of convolution branch in the i-th target feature processor.

[0013] Optionally, in the process of processing the training image in sequence through N feature processors in the neural network, the posture recognition method also includes: extracting at least one spatial feature corresponding to each object in the training image through a first type of convolution branch in the N feature processors; performing feature fusion and feature dimensionality reduction on at least one spatial feature corresponding to each object extracted by the first type of convolution branch through a second type of convolution branch in the neural network to obtain a target feature corresponding to each object, wherein the feature dimensionality reduction is used to ensure that the target feature is in the target dimension.

[0014] In order to achieve the above-mentioned purpose, according to another aspect of an embodiment of the present application, a posture recognition device is also provided, including: an acquisition unit, used to acquire a target image, wherein the target image includes at least one target object; an extraction unit, used to extract at least one spatial feature corresponding to each target object in the target image through a first type of convolution branch of a neural network; wherein the first type of convolution branch and the second type of convolution branch are connected in a parallel structure, and the number of channels of the second type of convolution branch is less than that of the first type of convolution branch, and the second type of convolution branch is used to perform feature fusion on at least one spatial feature corresponding to each target object extracted by the first type of convolution branch during the training stage of the neural network; a determination unit, used to determine the posture information of each target object according to at least one spatial feature corresponding to each target object in the target image.

[0015] Optionally, the extraction unit comprises N feature processors connected in series, wherein N is an integer greater than 1, and each of the N feature processors comprises a first type of convolution branch, a second type of convolution branch, and an activation function, wherein the activation function in each feature processor is used to learn the features output by the convolution branch in the feature processor.

[0016] Optionally, N feature processors connected in series include: an initial feature processor, wherein the initial feature processor is the first feature processor among the N feature processors, wherein the initial feature processor includes an initial convolution kernel composed of a first type of convolution branch and a second type of convolution branch, and the initial feature processor also includes an activation function connected in series with the initial convolution kernel.

[0017] Optionally, the N feature processors connected in series also include: N-1 target feature processors, wherein the target feature processor is a feature processor other than the initial feature processor among the N feature processors, the target feature processor includes a target convolution kernel composed of a first type of convolution branch, a second type of convolution branch and an identity branch, the target feature processor also includes an activation function connected in series with the target convolution kernel, and the identity branch of each target feature processor is used to input the output result of the feature processor of the previous layer corresponding to the target feature processor into the activation function of the target feature processor.

[0018] Optionally, the extraction unit further includes: an acquisition subunit and an iteration subunit. The acquisition unit is used to acquire a training image and label information corresponding to the training image, wherein the training image includes at least one object, and the label information is used to annotate the actual posture information of each object in the training image; the iteration subunit is used to perform multiple iterations of training on the neural network according to the training image and the label information, wherein the iteration training is used to update the weight parameters of the first and second convolution branches of the initial feature processor in the neural network, and / or update the weight parameters of the first, second and identical convolution branches in the neural network.

[0019] Optionally, the iteration subunit includes: a processing module and an updating module. The processing module is used to input the training image into the neural network, and process the training image in sequence through the N feature processors in the neural network; the updating module is used to update the neural network according to the output result and label information of the last target feature processor in the neural network after processing the training image.

[0020] Optionally, the processing module further includes a detection submodule, an input submodule 1, and an input submodule 2. The detection submodule is used to detect whether the identity branch and the first type of convolution branch in the i-th target feature processor have the same number of channels and step length, wherein i is a positive integer less than or equal to N-1; the input submodule 1 is used to input the output result of the feature processor of the previous level of the i-th target feature processor to the activation function in the i-th target feature processor through the identity branch in the i-th target feature processor when it is detected that the identity branch and the first type of convolution branch in the i-th target feature processor have the same number of channels and step length; the input submodule 2 is used to input the output result of the feature processor of the previous level of the i-th target feature processor to the activation function in the i-th target feature processor through the first type of convolution branch and the second type of convolution branch in the i-th target feature processor when it is detected that the identity branch and the first type of convolution branch in the i-th target feature processor do not have the same number of channels and step length.

[0021] Optionally, the processing module further includes an extraction submodule and a feature processing submodule. The extraction submodule is used to extract at least one spatial feature corresponding to each object in the training image through the first type of convolution branches in the N feature processors; the feature processing submodule is used to perform feature fusion and feature dimensionality reduction on at least one spatial feature corresponding to each object extracted by the first type of convolution branches through the second type of convolution branches in the neural network to obtain the target feature corresponding to each object, wherein the feature dimensionality reduction is used to ensure that the target feature is in the target dimension.

[0022] According to another aspect of an embodiment of the present application, an electronic device is further provided, including: a memory storing an executable program; and a processor for running the program, wherein the above-mentioned gesture recognition method is executed when the program is running.

[0023] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned gesture recognition method.

[0024] According to another aspect of an embodiment of the present application, a computer program product is also provided, including computer instructions, which implement the steps of the above-mentioned gesture recognition method when executed by a processor.

[0025] In an embodiment of the present application, a target image is first acquired, wherein the target image includes at least one target object, and then at least one spatial feature corresponding to each target object in the target image is extracted through a first type of convolution branch of a neural network; wherein the first type of convolution branch and the second type of convolution branch are connected in a parallel structure, and the number of channels of the second type of convolution branch is less than that of the first type of convolution branch, and the second type of convolution branch is used to perform feature fusion on at least one spatial feature corresponding to each target object extracted by the first type of convolution branch during the training stage of the neural network, and finally determine the posture information of each target object according to at least one spatial feature corresponding to each target object in the target image.

[0026] As can be seen from the above content, the present application first obtains a target image from a continuous video stream or an independent image, and the image contains at least one target object whose posture needs to be recognized.

[0027] Secondly, the first convolution branch of the neural network is used to extract the spatial features of each target object in the target image. This type of convolution branch is designed to capture the detailed texture and shape information of the target object, laying the foundation for subsequent posture recognition. The second type of convolution branch is connected in parallel with the first type of convolution branch and has a smaller number of channels. The purpose of this design is to effectively fuse the multiple spatial features extracted by the first type of convolution branch during the training phase of the neural network. The second type of convolution branch helps the network learn and adjust faster by reducing the computational complexity while maintaining the richness of the features, thereby improving real-time performance.

[0028] In addition, after extracting and fusing the spatial features, the corresponding algorithm is used to determine the posture information of each target object based on these features. The posture information includes but is not limited to the position, direction, action, etc. of the target object, which can help understand the activity status of the person in the image.

[0029] It can be seen that through the technical solution of the present application, the purpose of identifying the object posture is achieved by using a unique parallel convolutional architecture and an optimized training process, thereby achieving the technical effect of improving the real-time and accuracy of posture recognition, and further solving the technical problem of slow detection speed of object posture in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0031] Figure 1 A hardware structure block diagram of a computer terminal for implementing a gesture recognition method is shown;

[0032] Figure 2is a flowchart of an optional gesture recognition method according to an embodiment of the present application;

[0033] Figure 3 is an optional Rep Conv schematic diagram according to an embodiment of the present application;

[0034] Figure 4 is a schematic diagram of an optional EfficientRep structure according to an embodiment of the present application;

[0035] Figure 5 is a schematic diagram of an optional FocalNeXt structure according to an embodiment of the present application;

[0036] Figure 6 is a schematic diagram of a gesture recognition device provided according to an embodiment of the present application;

[0037] Figure 7 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0039] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0040] It should also be noted that the collected information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with relevant laws, regulations and standards, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation entrances are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions to provide users with corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, the expert decision-making process will be entered.

[0041] According to an embodiment of the present application, a method embodiment of a gesture recognition method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] It should be noted that a gesture recognition system can be used as the execution subject of the gesture recognition method of the embodiment of the present application. It is understandable that the gesture recognition method provided in the embodiment of the present application can also be executed by other systems or devices, and the embodiment of the present application does not specifically limit this.

[0043] According to an embodiment of the present application, a method embodiment of a gesture recognition method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a gesture recognition method. Figure 1As shown, the computer terminal 10 (or mobile device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0045] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuit may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As in the gesture recognition method involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0046] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the gesture recognition method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned gesture recognition method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0047] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0048] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0049] Under the above operating environment, this application provides Figure 2 The gesture recognition method shown. Figure 2 is a flow chart of an optional gesture recognition method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0050] Step S201: Acquire a target image, wherein the target image includes at least one target object.

[0051] Optionally, the target object is an object entity that needs to be identified and analyzed in the target image, and the target object may be a person, an animal, or other elements with dynamic behavior. In the present invention, it particularly refers to a person or an entity that can be regarded as a "person", and these objects may present a variety of postures in the image, including but not limited to standing, walking, running, jumping, or other more complex actions.

[0052] Optionally, the target image has specific context and requirements. It is not just a collection of pixels, but also a carrier of dynamic scene information, in which multiple target objects may be embedded, each of which may be in a different posture, thus forming a complex and rich visual information map.

[0053] Optionally, the process of acquiring the target image is not only an ordinary image reading action, but also a purposeful and intelligent image capture process designed to ensure the quality and relevance of the input data. Whether in a bank branch monitored by a fixed camera or in an outdoor scene captured by a mobile device, the step of acquiring the target image requires selecting images containing valuable information, that is, images with at least one target object.

[0054] Step S202: extract at least one spatial feature corresponding to each target object in the target image through a first type of convolution branch of the neural network.

[0055] In step S202, the first type of convolution branch is connected to the second type of convolution branch in a parallel structure, and the number of channels of the second type of convolution branch is less than that of the first type of convolution branch. The second type of convolution branch is used to perform feature fusion on at least one spatial feature corresponding to each target object extracted by the first type of convolution branch during the training stage of the neural network.

[0056] Alternatively, a neural network is a powerful deep learning model that draws on the working principles of brain neurons to understand and process visual information through complex network structures and algorithms. It is responsible for extracting meaningful features from raw image data and converting these features into high-level representations that can be used for gesture recognition through a series of carefully designed processing units. Each iteration and learning of the neural network makes it more sophisticated, able to accurately capture and understand the subtle movements of the target object, and can quickly and accurately interpret behavioral clues in the scene.

[0057] Optionally, the first type of convolution branch has rich channels and can capture the fine texture and complex structural information in the target image. Through this branch, the neural network can go deep into every corner of the image and extract the spatial features of the target object.

[0058] Optionally, the second type of convolution branch, although the number of channels is relatively streamlined, plays a great role in feature fusion. The second type of convolution branch cleverly runs in parallel with the first type of convolution branch, responsible for fusing the extracted spatial features during the training phase, reducing data redundancy, and enhancing the versatility of features and the robustness of the network.

[0059] Optionally, the gesture recognition system extracts at least one spatial feature corresponding to each target object in the target image through the first type of convolution branch of the neural network. The gesture recognition system extracts spatial features such as texture, shape and position of the target object through sophisticated convolution operations, so that the neural network can construct a three-dimensional image of the target object in the image, providing a rich and accurate data foundation for high-precision gesture recognition.

[0060] Step S203: determining the posture information of each target object according to at least one spatial feature corresponding to each target object in the target image.

[0061] Optionally, spatial features cover the shape, size, position and relative relationship of the target object to the surrounding environment, which is not only an abstract expression of data but also the cornerstone of visual understanding.

[0062] Optionally, posture information is a precise catalog of the target object's body posture and motion state at a specific moment. Posture information not only includes the target object's physical posture, such as standing, sitting, lying, and movement direction, but also reflects the target object's potential intentions and emotional state in more depth. In application scenarios such as bank monitoring, posture information can reflect suspicious behaviors to a certain extent, such as theft and property destruction, thereby providing an immediate response for security precautions.

[0063] Optionally, the posture recognition system converts the abstract spatial features of each target object in the target image into intuitive and clear posture information. The posture recognition system can convert the subtle differences in spatial features into accurate judgments of posture information through complex mathematical models, and can obtain the dynamic posture of the target object in the image, providing solid data support for subsequent behavior analysis and safety decision-making.

[0064] In an embodiment of the present application, a target image is first acquired, wherein the target image includes at least one target object, and then at least one spatial feature corresponding to each target object in the target image is extracted through a first type of convolution branch of a neural network; wherein the first type of convolution branch and the second type of convolution branch are connected in a parallel structure, and the number of channels of the second type of convolution branch is less than that of the first type of convolution branch, and the second type of convolution branch is used to perform feature fusion on at least one spatial feature corresponding to each target object extracted by the first type of convolution branch during the training stage of the neural network, and finally determine the posture information of each target object according to at least one spatial feature corresponding to each target object in the target image.

[0065] As can be seen from the above content, the present application first obtains a target image from a continuous video stream or an independent image, and the image contains at least one target object whose posture needs to be recognized.

[0066] Secondly, the first convolution branch of the neural network is used to extract the spatial features of each target object in the target image. This type of convolution branch is designed to capture the detailed texture and shape information of the target object, laying the foundation for subsequent posture recognition. The second type of convolution branch is connected in parallel with the first type of convolution branch and has a smaller number of channels. The purpose of this design is to effectively fuse the multiple spatial features extracted by the first type of convolution branch during the training phase of the neural network. The second type of convolution branch helps the network learn and adjust faster by reducing the computational complexity while maintaining the richness of the features, thereby improving real-time performance.

[0067] In addition, after extracting and fusing the spatial features, the corresponding algorithm is used to determine the posture information of each target object based on these features. The posture information includes but is not limited to the position, direction, action, etc. of the target object, which can help us understand the activity status of the person in the image.

[0068] It can be seen that through the technical solution of the present application, the purpose of identifying the object posture is achieved by using a unique parallel convolutional architecture and an optimized training process, thereby achieving the technical effect of improving the real-time and accuracy of posture recognition, and further solving the technical problem of slow detection speed of object posture in the prior art.

[0069] In an optional embodiment, the neural network includes: N feature processors connected in series, wherein N is an integer greater than 1, each of the N feature processors includes a first type of convolution branch, a second type of convolution branch, and an activation function, wherein the activation function in each feature processor is used to learn the features output by the convolution branch in the feature processor.

[0070] Alternatively, convolution is a mathematical operator that generates a third function from two functions. It is usually expressed as the area of ​​the curved trapezoid enclosed by the product function of the function f and the flipped and translated g.

[0071] Optionally, the present application adopts a network structure improved based on the RepVGG (Reparameterization Vector-Gated Convolutional Neural Network) network, which is unique in that a 3×3 convolution layer (i.e., the first type of convolution branch) is followed by a ReLU (Rectified Linear Unit) activation function, thereby being able to efficiently utilize hardware computing resources.

[0072] Optionally, the feature processor is a core component in the recognition process, including the first-class convolution branch, the second-class convolution branch and the activation function. It adopts an advanced parallel convolution branch structure, which can capture the details of the target object from different angles at the same time, and builds a solid bridge for subsequent complex calculations and precise recognition.

[0073] Optionally, N feature processors connected in series form an intelligent pipeline, which delivers and deepens the spatial features of the target object one by one. The series connection between feature processors realizes the continuous delivery and optimization of features.

[0074] Optionally, each feature processor includes a first-class convolution branch, a second-class convolution branch, and an activation function to extract the spatial features of the target object from different levels. The first-class convolution branch is responsible for mining fine features, while the second-class convolution branch focuses on feature fusion and optimization. The activation function introduces nonlinear transformations, allowing the network to learn richer and more complex data representations.

[0075] Optionally, in each feature processor, the activation function plays a key role in feature learning and optimization. Through clever nonlinear transformations, the features are guided to evolve in a more expressive and discriminative direction. This learning mechanism of the activation function ensures that the neural network can extract the most essential target object features from massive image data, providing a solid foundation for subsequent posture information determination.

[0076] From the above content, it can be seen that the posture recognition system combines the precise capture of the first type of convolution branches, the efficient fusion of the second type of convolution branches, and the intelligent optimization of the activation function through N feature processors connected in series. It not only improves the recognition accuracy of the neural network for the object posture in the target image, but also enhances its ability to process and adapt to complex scenes in real time.

[0077] In an optional embodiment, the neural network includes N feature processors connected in series, including: an initial feature processor, wherein the initial feature processor is the first feature processor among the N feature processors, wherein the initial feature processor includes an initial convolution kernel composed of a first type of convolution branch and a second type of convolution branch, and the initial feature processor also includes an activation function connected in series with the initial convolution kernel.

[0078] Optionally, the initial feature processor is the first feature processor among N feature processors, including an initial convolution kernel consisting of a first type of convolution branch and a second type of convolution branch, and an activation function connected in series with the initial convolution kernel, to provide a more suitable input for subsequent feature processors.

[0079] Optionally, the convolution kernel extracts the spatial information of the target object from each pixel of the image by sliding calculation. The size and shape of the convolution kernel determine the type and range of features it can capture. In this application, a convolution kernel composed of the first type of convolution branch and the second type of convolution branch is used. This design can not only capture the local details of the target object, but also maintain the efficiency of calculation and the lightweight of the network.

[0080] From the above content, we can see that the posture recognition system improves the recognition accuracy of the neural network for the object posture in the target image through N feature processors connected in series, and also enhances its ability to process and adapt to complex scenes in real time. The initial feature processor, that is, the first feature processor among the N feature processors, extracts the basic spatial features in the image, which constitute the cornerstone of subsequent analysis and recognition.

[0081] In an optional embodiment, the N feature processors connected in series included in the neural network also include: N-1 target feature processors, wherein the target feature processor is a feature processor other than the initial feature processor among the N feature processors, the target feature processor includes a target convolution kernel composed of a first type of convolution branch, a second type of convolution branch and an identity branch, the target feature processor also includes an activation function connected in series with the target convolution kernel, and the identity branch of each target feature processor is used to input the output result of the feature processor of the previous layer corresponding to the target feature processor into the activation function of the target feature processor.

[0082] Optionally, the target feature processor is a feature processor among the N feature processors except the initial feature processor, including a target convolution kernel composed of a first-type convolution branch, a second-type convolution branch and an identity branch, and also includes an activation function connected in series with the target convolution kernel, so as to further refine and extract high-level features of the target object on the basis of the initial feature processor.

[0083] Optionally, the role of the identity branch is to pass the input directly to the output. By directly adding the input to the output of the convolutional layer, a more direct gradient flow channel can be created, which helps to train deeper networks; at the same time, it can enable the network to better retain input information and avoid information loss or attenuation during multi-layer convolution.

[0084] Optionally, the target convolution kernel can observe the target object from different angles and scales to extract its deep spatial features. Compared with the convolution kernel in the initial feature processor, the target convolution kernel is more focused on extracting high-level features of the target object. Through more sophisticated convolution operations, the network can capture subtle changes in the target object in complex scenes, such as fine-tuning of posture and conversion of gestures.

[0085] From the above content, it can be seen that the target feature processor of the gesture recognition system ensures that the neural network can capture subtle changes of the target object in complex scenes, such as fine-tuning of the posture, conversion of gestures, etc. Even when the target object is partially occluded or the posture changes, its behavioral intention can be accurately identified and understood.

[0086] In an optional embodiment, the training process of the neural network includes the following steps: obtaining a training image and label information corresponding to the training image, wherein the training image includes at least one object, and the label information is used to annotate the actual posture information of each object in the training image; performing multiple iterative training on the neural network based on the training image and the label information, wherein the iterative training is used to update the weight parameters of the first type of convolution branch and the second type of convolution branch of the initial feature processor in the neural network, and / or, update the weight parameters of the first type of convolution branch, the second type of convolution branch and the identity branch in the neural network.

[0087] Alternatively, the training process of a neural network is a process of extracting wisdom from massive data and evolving continuously. Before training begins, a large number of training images and corresponding label information need to be prepared. These images and labels constitute the basic materials for neural network learning (i.e., the acquired target image data).

[0088] Optionally, in the training state, RepVGG consists of a 3×3 branch (i.e., the first type of convolution branch), a 1×1 branch (i.e., the second type of convolution branch), and an identity branch. In this structure, the additional 1×1 convolution (i.e., the second type of convolution branch) and the identity mapping ensure the accuracy of the model during training.

[0089] Optionally, during training, the identity branch works in parallel with other branches (such as 1x1 convolution branch and 3x3 convolution branch) to jointly participate in the forward propagation and backward propagation of the network, so that the model can learn more feature representations during the training process, increasing the complexity of the model.

[0090] Optionally, the process of acquiring training images is the first step in building a behavior recognition library, which involves collecting image data containing normal and abnormal behaviors from monitoring scenes such as bank branches. The posture recognition system uses professional monitoring equipment to record various behavior scenes to ensure the diversity and representativeness of the training images so that the neural network can fully grasp the behavior of the target object in different situations.

[0091] Optionally, the gesture recognition system needs to define the corresponding label information for each training image. The gesture recognition system classifies and labels the training images through manual labeling or using an existing behavior database, and clearly indicates whether the behavior in the image is an abnormal behavior such as theft or property damage. This provides a clear learning goal for the neural network, enabling it to gradually adjust and optimize its behavior recognition ability through a feedback mechanism.

[0092] Optionally, the training process of the neural network is a multi-iteration process. The posture recognition system updates the weight parameters in the network based on the feedback of the training image and label information through the back-propagation algorithm. In each iteration, the neural network forward-propagates the input training image to generate a prediction result, and then compares the prediction result with the label information to calculate the loss function to evaluate the prediction accuracy of the network. Next, through back-propagation, the neural network adjusts the weight parameters of the first-class convolution branch, the second-class convolution branch, and the identity branch within it according to the gradient of the loss function.

[0093] Optionally, the gesture recognition system uses the Intersection over Union (IoU) loss function to measure the overlap between the predicted box and the true box, which determines the positioning performance of the model. The WIoU function uses a dynamic adjustment gradient allocation strategy based on IoU to narrow the penalty gap between anchor boxes of different sizes, so that it performs better when facing targets with different geometric factors such as distances and aspect ratios, thereby improving the generalization performance of the model.

[0094] From the above content, we can see that the posture recognition system first obtains the training images and the label information corresponding to the training images, which provides a clear learning goal for the neural network, so that it can gradually adjust and optimize its behavior recognition ability through the feedback mechanism. During the iterative training process, the dynamic weight parameter update mechanism ensures that the neural network can intelligently adapt to the visual characteristics of different behaviors and improve its behavior recognition ability in complex scenes. The first and second convolution branches are responsible for feature extraction of different tasks, while the identity branch retains the original information of the features. By dynamically adjusting the weight parameters of these branches, the neural network can more accurately capture and understand the behavior patterns of the target objects, reduce misjudgments, and improve the accuracy and stability of recognition.

[0095] In an optional embodiment, iterative training includes the following steps: inputting a training image into a neural network, processing the training image in sequence through N feature processors in the neural network; and updating the neural network based on the output result and label information of the last target feature processor in the neural network after processing the training image.

[0096] Optionally, iterative training is the core process of neural network learning and optimizing its behavior recognition ability. Neural networks also need countless data inputs and feedback to gradually improve their recognition accuracy and generalization ability. This process not only involves data processing and network prediction, but also requires intelligent updating of network weights based on the difference between the prediction results and the true labels.

[0097] Optionally, at the beginning of training, images are selected from the constructed training image library and input into the neural network. Each training image is the starting point for neural network learning, and contains the behavioral information of the target object and the details of the environmental background. Through this process, the neural network begins to try to understand the content in the image, from simple edges and textures to complex postures and movements, and gradually builds up the perception and understanding of the target behavior.

[0098] Optionally, when a training image is input, the image information data will pass through N feature processors in the neural network in turn. Each processor has its own specific task, from extracting basic features to building advanced features, gradually deepening the understanding of the target behavior. The feature processor uses convolution kernels to operate, and through operations such as convolution, pooling, and nonlinear transformation, it extracts features that are critical to identifying the target behavior.

[0099] Optionally, after being processed by N feature processors, the target feature processor at the last level will output the final feature representation of the training image by the neural network. Based on these features, the neural network will generate a prediction result for the target behavior.

[0100] From the above content, it can be seen that the posture recognition system can gradually reduce the prediction error and improve the accuracy of behavior recognition through input processing of training images, step-by-step operation of feature processors in the neural network, and continuous iteration and adjustment of the neural network based on output results and label information. It will eventually become an intelligent system that can accurately identify abnormal behaviors such as theft and damage to property in complex environments such as bank branches.

[0101] In an optional embodiment, in the process of sequentially processing the training image through N feature processors in the neural network, the posture recognition method further includes: detecting whether the identity branch and the first type of convolution branch in the i-th target feature processor have the same number of channels and step length, wherein i is a positive integer less than or equal to N-1; when it is detected that the identity branch and the first type of convolution branch in the i-th target feature processor have the same number of channels and step length, the output result of the feature processor of the previous level of the i-th target feature processor is input into the activation function in the i-th target feature processor through the identity branch in the i-th target feature processor; when it is detected that the identity branch and the first type of convolution branch in the i-th target feature processor do not have the same number of channels and step length, the output result of the feature processor of the previous level of the i-th target feature processor is input into the activation function in the i-th target feature processor through the first type of convolution branch and the second type of convolution branch in the i-th target feature processor.

[0102] Optionally, the gesture recognition system detects whether the identity branch and the first-class convolution branch in the i-th target feature processor have the same number of channels and stride, which is an important check in the neural network architecture design. The number of channels determines the dimension of information, and the stride affects the speed and accuracy of information transmission. The consistency of the number of channels and stride of the identity branch and the first-class convolution branch ensures that the information flow inside the feature processor can be transmitted unimpeded.

[0103] Optionally, when the gesture recognition system detects that the identity branch and the first type convolution branch have the same number of channels and stride, it means that the coordination mechanism of the information flow inside the feature processor of the neural network works properly. In this case, the feature processor can smoothly fuse the information of the identity branch with the information processed by the first type convolution branch to form a richer visual feature representation.

[0104] Optionally, when it is detected that the number of channels or the stride of the identity branch and the first type of convolution branch are inconsistent, the neural network adopts a corresponding strategy to adjust the mismatch to keep the information flow unimpeded. To avoid this, the gesture recognition system ensures the consistency of the identity branch and the first type of convolution branch in terms of dimension and stride.

[0105] From the above content, it can be seen that the posture recognition system detects whether the identity branch and the first type of convolution branch in the i-th target feature processor have the same number of channels and step size, which is the key to ensuring the coordination of the internal information flow of the neural network, enabling the neural network to overcome the mismatch problem of the internal information flow of the feature processor, maintain the stability and efficiency of the network, and ensure excellent performance in the behavior recognition task.

[0106] In an optional embodiment, in the process of processing the training image in sequence through N feature processors in the neural network, the posture recognition method also includes: extracting at least one spatial feature corresponding to each object in the training image through a first type of convolution branch in the N feature processors; performing feature fusion and feature dimensionality reduction on at least one spatial feature corresponding to each object extracted by the first type of convolution branch through a second type of convolution branch in the neural network to obtain a target feature corresponding to each object, wherein the feature dimensionality reduction is used to ensure that the target feature is in the target dimension.

[0107] Optionally, in the N feature processors in the neural network, the first type of convolution branch can extract key spatial features from each object by performing dense convolution operations on the training image. These spatial features include but are not limited to the shape, posture and relative position of the object to other objects.

[0108] Optionally, feature fusion refers to combining feature information obtained from different sources or through different processing paths to form a richer and more comprehensive high-level feature representation.

[0109] Optionally, feature dimensionality reduction refers to the process of converting a high-dimensional feature space into a low-dimensional feature space, with the goal of reducing the number of features while retaining the most important information to reduce the computational complexity of the model and avoid overfitting.

[0110] Optionally, the main task of the second type of convolution branch is to perform feature fusion and dimensionality reduction on at least one spatial feature corresponding to each object extracted by the first type of convolution branch, to ensure that the neural network can efficiently process and understand these features and avoid information redundancy and over-complexity. Feature fusion means combining the spatial features of different objects together to form a more comprehensive and abstract feature representation, which helps the neural network understand the relationship between objects and the context of behavior. Feature dimensionality reduction is to reduce the dimension of the feature through a 1×1 convolution layer or other linear transformation methods, retain the most critical information, while reducing the computational cost and increasing the running speed of the model.

[0111] From the above content, it can be seen that the posture recognition system first extracts at least one spatial feature corresponding to each object in the training image through the first type of convolution branches in the N feature processors. Then, the second type of convolution branches in the neural network perform feature fusion and feature dimensionality reduction on the at least one spatial feature corresponding to each object extracted by the first type of convolution branches, which can reduce the consumption of computing resources and improve the generalization ability of the model, so that it can still accurately identify new and unknown behavior patterns.

[0112] It can be seen from the above content that according to the technical solution of this application, at least the following technical effects can be achieved:

[0113] 1. Effectively solve the real-time problem and improve the accuracy.

[0114] 2. Reduce the possibility and problem of missed detection in severe occlusion and crowded conditions.

[0115] 3. Through more accurate and real-time monitoring algorithms, theft and property damage behaviors in outlets can be identified.

[0116] In an optional embodiment, Figure 3 A gesture recognition method according to an embodiment of the present application is shown. Figure 3 An optional Rep Conv instance shown in the figure introduces the EfficientRep network to replace the YOLOv8 backbone network to meet real-time requirements. The efficient computing resource utilization of the EfficientRep network significantly improves the speed and accuracy of target detection, enabling the algorithm to respond instantly to different changes in human posture, effectively solving the real-time problem. RepVGG is an innovative network structure. Its uniqueness lies in a 3×3 convolution layer followed by a ReLU activation function, which can efficiently utilize hardware computing resources. In the training state, RepVGG consists of a 3×3 branch, a 1×1 branch, and an identity branch. Under this structure, the additional 1×1 convolution and identity mapping ensure the accuracy of the model during training. The RepVGG structure diagram is shown in the figure. Figure 3As shown. However, in the inference state, the structure will be reparameterized to equivalently convert to the inference state. The structural reparameterization method is inspired by ResNet, which explicitly constructs a shortcut branch to model the information flow as y=x+f(x)+g(x) and uses a residual block to learn f, where x, g(x), and f(x) are the identity branch, 1×1 convolution, and 3X3 convolution, respectively. When the dimensions of x and f(x) do not match, it becomes y=g(x)+f(x), where g(x) is a convolution shortcut implemented by 1×1conv. This design allows RepVGG to remain efficient and accurate in both training and inference states.

[0117] In an optional embodiment, Figure 4 A gesture recognition method according to an embodiment of the present application is shown. Figure 4 An optional EfficientRep instance is shown, with ResNet, RepVGG training state, and RepVGG inference state from left to right. Based on the basic RepVGG, a pure RepVGG-style architecture is designed, named EfficientRep network. The main body of RepVGG at reasoning time has only one type of operator: 3×3 convolution followed by ReLU, which makes RepVGG have a favorable speed-accuracy trade-off. During the training phase of RepVGG, each RepVGG block contains a convolution shortcut branch, and if the dimensions of the input and output match (that is, the number of channels is the same and the stride is 1), the input can be directly added to the output.

[0118] In an optional embodiment, Figure 5 A gesture recognition method according to an embodiment of the present application is shown. Figure 5 An optional FocalNeXt instance is shown in FIG. 1 , wherein the FocalNeXt structure is formed by stacking multiple FocalBlock modules, such as Figure 5As shown. By stacking FocalBlock modules multiple times, the FocalNeXt module is able to perform deep feature extraction and transformation in the network, further enhancing the expressiveness and performance of the model. The FocalBlock module inside FocalNeXt carefully combines different operations to reduce the number of parameters and computational costs while maintaining high performance. Among them, RepConv is the convolutional layer used in the RepVGG model. It uses a multi-branch structure during the training phase, including convolution kernels of different sizes (such as 3x3 convolution, 1x1 convolution, and identity mapping branches) to enhance the expressiveness of the model. The stride is set to 2 to gradually reduce the spatial dimension while increasing the channel dimension to capture more abstract features. RepBlock is a core component in the RepVGG model and contains multiple convolutional layers. SIMSPPF is an important module used to enhance the model's ability to extract features of different scales.

[0119] Based on the two structures of FocaNeXt and EfficientRep, EfficientRep is suitable for scenarios that require efficient reasoning, and FocalNeXt is suitable for visual tasks that pursue high performance. It can achieve high-precision detection results while maintaining a fast reasoning speed. In addition, Focal Block is a core module in the FocalNeXt network. It uses the FocalModulation mechanism to enhance feature representation and improve the model's ability to capture local and global contextual information. FocalBlock usually contains the following components: Convolutional layer: used for initial feature extraction. Focal Modulation: modulates the initial features to enhance feature representation. Fusion layer: fuses the modulated features with the original features.

[0120] According to another aspect of the embodiment of the present application, a gesture recognition device is also provided, wherein: Figure 6 is a schematic diagram of an optional gesture recognition device according to an embodiment of the present application, such as Figure 6 As shown, the gesture recognition device includes: an acquisition unit 601 , an extraction unit 602 , and a determination unit 603 .

[0121] An acquisition unit 601 is used to acquire a target image, wherein the target image includes at least one target object; an extraction unit 602 is used to extract at least one spatial feature corresponding to each target object in the target image through a first type of convolution branch of a neural network; wherein the first type of convolution branch and the second type of convolution branch are connected in a parallel structure, and the number of channels of the second type of convolution branch is less than that of the first type of convolution branch, and the second type of convolution branch is used to perform feature fusion on at least one spatial feature corresponding to each target object extracted by the first type of convolution branch during the training phase of the neural network; a determination unit 603 is used to determine the posture information of each target object according to at least one spatial feature corresponding to each target object in the target image.

[0122] Optionally, the extraction unit 602 includes N feature processors connected in series, where N is an integer greater than 1, and each of the N feature processors includes a first type of convolution branch, a second type of convolution branch, and an activation function, wherein the activation function in each feature processor is used to learn the features output by the convolution branch in the feature processor.

[0123] Optionally, N feature processors connected in series include: an initial feature processor, wherein the initial feature processor is the first feature processor among the N feature processors, wherein the initial feature processor includes an initial convolution kernel composed of a first type of convolution branch and a second type of convolution branch, and the initial feature processor also includes an activation function connected in series with the initial convolution kernel.

[0124] Optionally, the N feature processors connected in series also include: N-1 target feature processors, wherein the target feature processor is a feature processor other than the initial feature processor among the N feature processors, the target feature processor includes a target convolution kernel composed of a first type of convolution branch, a second type of convolution branch and an identity branch, the target feature processor also includes an activation function connected in series with the target convolution kernel, and the identity branch of each target feature processor is used to input the output result of the feature processor of the previous layer corresponding to the target feature processor into the activation function of the target feature processor.

[0125] Optionally, the extraction unit 602 further includes: an acquisition subunit and an iteration subunit. The acquisition unit is used to acquire a training image and label information corresponding to the training image, wherein the training image includes at least one object, and the label information is used to annotate the actual posture information of each object in the training image; the iteration subunit is used to perform multiple iterations of training the neural network according to the training image and the label information, wherein the iteration training is used to update the weight parameters of the first type of convolution branch and the second type of convolution branch of the initial feature processor in the neural network, and / or update the weight parameters of the first type of convolution branch, the second type of convolution branch and the identity branch in the neural network.

[0126] Optionally, the iteration subunit includes: a processing module and an updating module. The processing module is used to input the training image into the neural network, and process the training image in sequence through the N feature processors in the neural network; the updating module is used to update the neural network according to the output result and label information of the last target feature processor in the neural network after processing the training image.

[0127] Optionally, the processing module further includes a detection submodule, an input submodule 1, and an input submodule 2. The detection submodule is used to detect whether the identity branch and the first type of convolution branch in the i-th target feature processor have the same number of channels and step length, wherein i is a positive integer less than or equal to N-1; the input submodule 1 is used to input the output result of the feature processor of the previous level of the i-th target feature processor to the activation function in the i-th target feature processor through the identity branch in the i-th target feature processor when it is detected that the identity branch and the first type of convolution branch in the i-th target feature processor have the same number of channels and step length; the input submodule 2 is used to input the output result of the feature processor of the previous level of the i-th target feature processor to the activation function in the i-th target feature processor through the first type of convolution branch and the second type of convolution branch in the i-th target feature processor when it is detected that the identity branch and the first type of convolution branch in the i-th target feature processor do not have the same number of channels and step length.

[0128] Optionally, the processing module further includes an extraction submodule and a feature processing submodule. The extraction submodule is used to extract at least one spatial feature corresponding to each object in the training image through the first type of convolution branches in the N feature processors; the feature processing submodule is used to perform feature fusion and feature dimensionality reduction on at least one spatial feature corresponding to each object extracted by the first type of convolution branches through the second type of convolution branches in the neural network to obtain the target feature corresponding to each object, wherein the feature dimensionality reduction is used to ensure that the target feature is in the target dimension.

[0129] An embodiment of the present application may provide an electronic device, Figure 7 is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 7 As shown, the electronic device may include: one or more ( Figure 7 Only one is shown) a processor, a memory, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0130] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0131] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain a target image, wherein the target image includes at least one target object; extract at least one spatial feature corresponding to each target object in the target image through a first type of convolution branch of the neural network; wherein the first type of convolution branch and the second type of convolution branch are connected in a parallel structure, and the number of channels of the second type of convolution branch is less than that of the first type of convolution branch, and the second type of convolution branch is used to perform feature fusion on at least one spatial feature corresponding to each target object extracted by the first type of convolution branch during the training stage of the neural network; determine the posture information of each target object according to at least one spatial feature corresponding to each target object in the target image.

[0132] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: N feature processors connected in series, where N is an integer greater than 1, and each of the N feature processors includes a first type of convolution branch, a second type of convolution branch, and an activation function, wherein the activation function in each feature processor is used to learn the features output by the convolution branch in the feature processor.

[0133] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: an initial feature processor, wherein the initial feature processor is the first feature processor among N feature processors, wherein the initial feature processor includes an initial convolution kernel composed of a first type of convolution branch and a second type of convolution branch, and the initial feature processor also includes an activation function connected in series with the initial convolution kernel.

[0134] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: N-1 target feature processors, wherein the target feature processor is a feature processor other than the initial feature processor among the N feature processors, and the target feature processor includes a target convolution kernel composed of a first type of convolution branch, a second type of convolution branch and an identity branch, and the target feature processor also includes an activation function connected in series with the target convolution kernel, and the identity branch of each target feature processor is used to input the output result of the feature processor of the previous layer corresponding to the target feature processor into the activation function of the target feature processor.

[0135] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: obtain a training image and label information corresponding to the training image, wherein the training image includes at least one object, and the label information is used to mark the actual posture information of each object in the training image; perform multiple iterative training on the neural network according to the training image and the label information, wherein the iterative training is used to update the weight parameters of the first type of convolution branch and the second type of convolution branch of the initial feature processor in the neural network, and / or update the weight parameters of the first type of convolution branch, the second type of convolution branch and the identity branch in the neural network.

[0136] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: input the training image into the neural network, and process the training image in sequence through the N feature processors in the neural network; update the neural network according to the output result and label information of the last target feature processor in the neural network after processing the training image.

[0137] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: detect whether the identity branch and the first type of convolution branch in the i-th target feature processor have the same number of channels and step size, where i is a positive integer less than or equal to N-1; when it is detected that the identity branch and the first type of convolution branch in the i-th target feature processor have the same number of channels and step size, input the output result of the feature processor of the previous level of the i-th target feature processor into the activation function in the i-th target feature processor through the identity branch in the i-th target feature processor; when it is detected that the identity branch and the first type of convolution branch in the i-th target feature processor do not have the same number of channels and step size, input the output result of the feature processor of the previous level of the i-th target feature processor into the activation function in the i-th target feature processor through the first type of convolution branch and the second type of convolution branch in the i-th target feature processor.

[0138] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: extract at least one spatial feature corresponding to each object in the training image through the first type of convolution branches in the N feature processors; perform feature fusion and feature dimensionality reduction on the at least one spatial feature corresponding to each object extracted by the first type of convolution branches through the second type of convolution branches in the neural network to obtain the target feature corresponding to each object, wherein the feature dimensionality reduction is used to ensure that the target feature is in the target dimension.

[0139] In an embodiment of the present application, a target image is first acquired, wherein the target image includes at least one target object, and then at least one spatial feature corresponding to each target object in the target image is extracted through a first type of convolution branch of a neural network; wherein the first type of convolution branch and the second type of convolution branch are connected in a parallel structure, and the number of channels of the second type of convolution branch is less than that of the first type of convolution branch, and the second type of convolution branch is used to perform feature fusion on at least one spatial feature corresponding to each target object extracted by the first type of convolution branch during the training stage of the neural network, and finally determine the posture information of each target object according to at least one spatial feature corresponding to each target object in the target image.

[0140] As can be seen from the above content, the present application first obtains a target image from a continuous video stream or an independent image, and the image contains at least one target object whose posture needs to be recognized.

[0141] Secondly, the first convolution branch of the neural network is used to extract the spatial features of each target object in the target image. This type of convolution branch is designed to capture the detailed texture and shape information of the target object, laying the foundation for subsequent posture recognition. The second type of convolution branch is connected in parallel with the first type of convolution branch and has a smaller number of channels. The purpose of this design is to effectively fuse the multiple spatial features extracted by the first type of convolution branch during the training phase of the neural network. The second type of convolution branch helps the network learn and adjust faster by reducing the computational complexity while maintaining the richness of the features, thereby improving real-time performance.

[0142] In addition, after extracting and fusing the spatial features, the corresponding algorithm is used to determine the posture information of each target object based on these features. The posture information includes but is not limited to the position, direction, action, etc. of the target object, which can help us understand the activity status of the person in the image.

[0143] It can be seen that through the technical solution of the present application, the purpose of identifying the object posture is achieved by using a unique parallel convolutional architecture and an optimized training process, thereby achieving the technical effect of improving the real-time and accuracy of posture recognition, and further solving the technical problem of slow detection speed of object posture in the prior art.

[0144] It can be understood by those skilled in the art that Figure 7 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 7 The structure of the electronic device is not limited. Figure 7 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 7 Different configurations shown.

[0145] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0146] According to another aspect of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the above-mentioned gesture recognition method.

[0147] According to another aspect of the present application, a computer program product is also provided, wherein the computer program product includes computer instructions, wherein when the computer instructions are executed, the device where the computer program product is located executes the above-mentioned gesture recognition method.

[0148] The above-mentioned embodiments or examples disclosed in the present application are not exhaustive, but are only illustrative of some embodiments or examples, and are not intended to be specific limitations on the scope of protection disclosed in the present application. In the absence of contradiction, each step in a certain embodiment or example in the present application can be implemented as an independent example, and the steps can be combined arbitrarily. For example, the scheme after removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be arbitrarily exchanged. In addition, the optional methods or optional examples in a certain embodiment or example can be combined arbitrarily; in addition, the various embodiments or examples can be combined arbitrarily, for example, some or all steps of different embodiments or examples can be combined arbitrarily, and a certain embodiment or example can be combined arbitrarily with the optional methods or optional examples of other embodiments or examples.

[0149] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0150] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0152] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0153] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0154] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0155] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A gesture recognition method, characterized in that: include: Acquire a target image, wherein the target image includes at least one target object; Extracting at least one spatial feature corresponding to each target object in the target image through a first type of convolution branch of the neural network; wherein the first type of convolution branch is connected to a second type of convolution branch in a parallel structure, and the number of channels of the second type of convolution branch is less than that of the first type of convolution branch, and the second type of convolution branch is used to perform feature fusion on at least one spatial feature corresponding to each target object extracted by the first type of convolution branch during the training phase of the neural network; The posture information of each target object in the target image is determined according to at least one spatial feature corresponding to each target object.

2. The gesture recognition method according to claim 1, characterized in that: The neural network includes: N feature processors connected in series, wherein N is an integer greater than 1, and each of the N feature processors includes the first type of convolution branch, the second type of convolution branch and an activation function, wherein the activation function in each feature processor is used to learn the features output by the convolution branch in the feature processor.

3. The gesture recognition method according to claim 1, characterized in that: The neural network includes N feature processors connected in series, including: An initial feature processor, wherein the initial feature processor is the first feature processor among N feature processors, wherein the initial feature processor includes an initial convolution kernel composed of the first type of convolution branches and the second type of convolution branches, and the initial feature processor also includes an activation function connected in series with the initial convolution kernel.

4. The gesture recognition method according to claim 3, characterized in that: The neural network includes N series-connected feature processors and further includes: N-1 target feature processors, wherein the target feature processor is a feature processor among the N feature processors except the initial feature processor, the target feature processor includes a target convolution kernel composed of the first type of convolution branches, the second type of convolution branches and an identity branch, the target feature processor also includes an activation function connected in series with the target convolution kernel, and the identity branch of each target feature processor is used to input the output result of the feature processor of the previous layer corresponding to the target feature processor into the activation function of the target feature processor.

5. The gesture recognition method according to claim 4, characterized in that: The training process of the neural network includes the following steps: Acquire a training image and label information corresponding to the training image, wherein the training image includes at least one object, and the label information is used to annotate actual posture information of each object in the training image; The neural network is iteratively trained for multiple times according to the training image and the label information, wherein the iterative training is used to update the weight parameters of the first type of convolution branch and the second type of convolution branch of the initial feature processor in the neural network, and / or to update the weight parameters of the first type of convolution branch, the second type of convolution branch and the identity branch in the neural network.

6. The gesture recognition method according to claim 5, characterized in that: The iterative training comprises the following steps: Inputting the training image into the neural network, and sequentially processing the training image through N feature processors in the neural network; The neural network is updated according to the output result of the last target feature processor in the neural network after processing the training image and the label information.

7. The gesture recognition method according to claim 6, characterized in that: In the process of sequentially processing the training images by the N feature processors in the neural network, the gesture recognition method further includes: Check whether the identity branch and the first type convolution branch in the i-th target feature processor have the same number of channels and stride, where i is a positive integer less than or equal to N-1; When it is detected that the identity branch in the i-th target feature processor and the first type of convolution branch have the same number of channels and step size, an output result of the feature processor at the previous level of the i-th target feature processor is input into the activation function in the i-th target feature processor through the identity branch in the i-th target feature processor; When it is detected that the identity branch and the first type of convolution branch in the i-th target feature processor do not have the same number of channels and step size, the output result of the feature processor at the previous level of the i-th target feature processor is input into the activation function in the i-th target feature processor through the first type of convolution branch and the second type of convolution branch in the i-th target feature processor.

8. The gesture recognition method according to claim 6, characterized in that: In the process of sequentially processing the training images by the N feature processors in the neural network, the gesture recognition method further includes: Extracting at least one spatial feature corresponding to each object in the training image through a first type of convolution branch in the N feature processors; The second type of convolution branch in the neural network performs feature fusion and feature dimensionality reduction on at least one spatial feature corresponding to each object extracted by the first type of convolution branch to obtain a target feature corresponding to each object, wherein the feature dimensionality reduction is used to ensure that the target feature is in a target dimension.

9. A gesture recognition device, characterized in that: include: An acquisition unit, configured to acquire a target image, wherein the target image includes at least one target object; An extraction unit, configured to extract at least one spatial feature corresponding to each target object in the target image through a first type of convolution branch of a neural network; wherein the first type of convolution branch is connected to a second type of convolution branch in a parallel structure, and the number of channels of the second type of convolution branch is less than that of the first type of convolution branch, and the second type of convolution branch is configured to perform feature fusion on at least one spatial feature corresponding to each target object extracted by the first type of convolution branch during a training phase of the neural network; A determination unit is used to determine the posture information of each target object in the target image according to at least one spatial feature corresponding to each target object.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute the gesture recognition method according to any one of claims 1 to 8.

11. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the program executes the gesture recognition method described in any one of claims 1 to 8 when running.

12. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the gesture recognition method described in any one of claims 1 to 8 are implemented.

Images