A data enhancement method and system for distinguishing the nature of crowd activities

Abstract

The present invention provides a data enhancement method and system for distinguishing the nature of crowd activities, including preparing a crowd activity training data set and a pre-training model for distinguishing the nature of crowd activities to generate a heat map; randomly extracting a data pair from the crowd activity training data set, using a pixel-level linear mixing enhancement strategy, and using linear combinations to mix images and labels; using a region-level affine splicing enhancement strategy to splice images through cut-and-paste operations, and mix labels according to area ratios; using an enhanced class gradient activation visualization strategy to extract the output class activation heat map, perform image secondary mixing enhancement and label fusion, and form a secondary mixed image enhanced data set to expand the original data set. This application effectively and specifically achieves the expansion of the relevant sample library, and its expansion process and results can have a significant positive impact on the crowd activity nature discrimination algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a data enhancement method and system for distinguishing the nature of crowd activities. Background Art

[0002] Crowd activity identification involves summarizing the semantic properties of large crowds in public scenes, thereby identifying the names and characteristics of the human activities in the image. These samples often exhibit undesirable characteristics such as fragmented semantics and unclear focus. While algorithms combining human object detection and semantic segmentation can count and locate individual people in an image, they still struggle to effectively integrate interrelated information between individuals for overall analysis. Crowd counting algorithms can effectively count the number of people in an image, but they still cannot accurately reflect the type of human activity they represent. There are also auxiliary solutions for analyzing secondary semantics in images, such as capturing overlapping features like slogans, glow sticks, and human arrangements. However, these approaches are complex, often employ heuristics, consume significant resources, and are subject to multiple influences, resulting in poor results. Classification models offer direct reasoning, ease of use, low resource consumption, and strong generalization, making them the preferred solution for this problem. However, they require a large sample size, resulting in unsatisfactory accuracy from direct training and prone to scene overfitting. Scene overfitting binds category attributes to specific scenes, severely misleading the classification results for such images. At present, no enhancement scheme has been found specifically for the algorithm for distinguishing the nature of crowd activities. Summary of the Invention

[0003] In order to solve the technical problem that there is no enhancement solution specifically for the crowd activity nature discrimination algorithm in the prior art, the present invention proposes a data enhancement method and system for crowd activity nature discrimination to solve the above technical problem.

[0004] According to a first aspect of the present invention, a data enhancement method for distinguishing the nature of crowd activities is proposed, comprising:

[0005] S1: Prepare crowd activity training dataset and pre-trained model for crowd activity nature identification to generate heatmap;

[0006] S2: Randomly extract a data pair from the crowd activity training dataset and use a pixel-level linear mixing enhancement strategy to mix the image and label using linear combination;

[0007] S3: Uses a region-level affine stitching enhancement strategy to stitch images together through cut-and-paste operations and mix labels based on area ratios;

[0008] S4: By strengthening the class gradient activation visualization strategy, extracting the output class activation heat map, performing image secondary hybrid enhancement and label fusion, and forming a secondary hybrid image enhancement dataset to expand the original dataset.

[0009] In some specific embodiments, the pre-training model includes xception or Senet, and the crowd activity training dataset is defined as {(I i , Y i )|i=0, 1, ....N-1}, where, I i ∈R 3 ×W×H is a standard RGB image, i For image labels.

[0010] In some specific embodiments, S2 is specifically as follows: randomly extract a data pair {(I1, Y1), (I2, Y2)} from the crowd activity training data set, set two parameters b1 and b2 to extract two pairs of proportional parameters (γ1, γ2), (γ3, γ40) from a Beta distribution Beta (b1, b2); use linear combination to mix images and labels: I M1 =γ1×T s (I1)+(1-γ1)×T s (I2); U a =γ1,U b =1-γ1;Y M1 =U a ×Y1+U b ×Y2; where I M1 is the mixed image, Y M1 is the corresponding mixed label, T s It is a random data enhancement function of the same type that meets the fusion morphology scale requirements.

[0011] In some specific embodiments, S3 is specifically expressed as: Q a =1-γ2,Q b =γ2;Y M2 =Q a ×Y1+Q b ×Y2; where I M2 is the stitched image, Y M2 is the corresponding mixed label, T s In order to meet the fusion morphology scale requirements, random data enhancement functions of the same type are used.

[0012] In some specific embodiments, S4 extracts the output class activation heat map by strengthening the class gradient activation visualization strategy, specifically expressed as: in represents the class activation heat map obtained for the Cth category, i, j represents the pixel coordinates, To activate the attention mask, is the adaptive coefficient, is the kth feature map, for L cUpsample to make it the same size as the input image and obtain right Perform semantic graph mapping so that the sum of its pixels is 1.

[0013] In some specific embodiments, the secondary hybrid enhancement of the image in S4 is specifically as follows: in and are two binary masks, containing random box regions with area ratio γ3 and random box regions with area ratio γ4, TR θ As the conversion function, I M2 The final cropped area is converted to match I M1 The box area; the label fusion method is: Y Mix =C a ×Y M1 +C b ×Y M2 , where C a , C b is the semantic weight of the secondary mixed label.

[0014] In some specific embodiments, the expansion ratio of the original data set in S4 is to generate 35% of the data using a pixel-level linear mixing enhancement strategy, generate 35% of the data using a region-level affine splicing enhancement strategy, and generate 30% of the data using image secondary mixing.

[0015] According to a second aspect of the present invention, a computer-readable storage medium is provided, on which one or more computer programs are stored. When the one or more computer programs are executed by a computer processor, any one of the above methods is implemented.

[0016] According to a third aspect of the present invention, a data enhancement system for distinguishing the nature of crowd activities is proposed, the system comprising:

[0017] Preparation unit: Configures a pre-trained model for preparing crowd activity training datasets and identifying the nature of crowd activity to generate heat maps;

[0018] Pixel-level linear mixing enhancement unit: This unit is configured to randomly extract a data pair from the crowd activity training dataset and use a pixel-level linear mixing enhancement strategy to mix the image and label using a linear combination.

[0019] Region-level affine stitching enhancement unit: configured to use the region-level affine stitching enhancement strategy to stitch images through cut-and-paste operations and mix labels according to area ratios;

[0020] Dataset expansion unit: It is configured to extract the output class activation heat map by strengthening the class gradient activation visualization strategy, perform image secondary hybrid enhancement and label fusion, and form a secondary hybrid image enhancement dataset to expand the original dataset.

[0021] In some specific embodiments, the pre-training model includes xception or Senet, and the crowd activity training dataset is defined as {(I i , Y i )|i=0, 1, ....N-1}, where, I i ∈R 3 ×W×H is a standard RGB image, i For image labels.

[0022] In some specific embodiments, the pixel-level linear mixing enhancement unit is specifically configured to: randomly extract a data pair {(I1, Y1), (I2, Y2)} from the crowd activity training data set, set two parameters b1 and b2 to extract two pairs of proportional parameters (γ1, γ2), (γ3, γ4) from a Beta distribution Beta (b1, b20); use linear combination to mix the image and label: I M1 =γ1×T s (I1)+(1-γ1)×T s (I2); U a =γ1,U b =1-γ1;Y 11 =U a ×Y1+U b ×Y2; where I M1 is the mixed image, Y M1 is the corresponding mixed label, T s It is a random data enhancement function of the same type that meets the fusion morphology scale requirements.

[0023] In some specific embodiments, the region-level affine stitching enhancement unit is specifically expressed as: Q a =1-γ2,Q b =γ2;Y M2 =Q a ×Y1+Q b ×Y2; where I M2 is the stitched image, Y M2 is the corresponding mixed label, T s In order to meet the fusion morphology scale requirements, random data enhancement functions of the same type are used.

[0024] In some specific embodiments, the dataset expansion unit is specifically configured to: extract the output class activation heat map by strengthening the class gradient activation visualization strategy, specifically expressed as: in represents the class activation heat map obtained for the Cth category, i, j represents the pixel coordinates, To activate the attention mask, is the adaptive coefficient, is the kth feature map, for L c Upsample to make it the same size as the input image and obtain right Perform semantic map mapping so that the sum of its pixels is 1; the image secondary mixing enhancement is specifically as follows: in and are two binary masks, containing random box regions with area ratio γ3 and random box regions with area ratio γ4, TR θ As the conversion function, I M2 The final cropped area is converted to match I M1 The box area; the label fusion method is: Y Mix =C a ×Y M1 +C b ×Y M2 , where C a , C b is the semantic weight of the secondary mixed label.

[0025] In some specific embodiments, the expansion ratio of the original data set is to generate 35% of the data using a pixel-level linear mixing enhancement strategy, generate 35% of the data using a region-level affine splicing enhancement strategy, and generate 30% of the data using image secondary mixing.

[0026] This paper proposes a data enhancement method and system for distinguishing the nature of crowd activities. It also proposes a novel crowd scene sample synthesis scheme for this data enhancement method, thereby effectively and specifically expanding the relevant sample library. Both the expansion process and results can have a significant positive impact on the crowd activity distinction algorithm. This is the first to address the problem of distinguishing the nature of crowd activities from the perspective of scene overfitting, focusing on data enhancement and rationalizing and normalizing the sample set distribution. This method addresses the problem from the root cause and achieves significant results. It can be adapted to any framework and any algorithm model. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and, together with the description, serve to explain the principles of the invention. Other embodiments and many of the intended advantages of the embodiments will be readily apparent as they become better understood by reference to the following detailed description. Other features, objects, and advantages of the present application will become more apparent upon reading the detailed description of the non-limiting embodiments made with reference to the following drawings:

[0028] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0029] Figure 2 This is a flow chart of a data enhancement method for distinguishing the nature of crowd activities according to an embodiment of the present application;

[0030] Figure 3 This is a framework diagram of a data enhancement system for distinguishing the nature of crowd activities according to an embodiment of the present application;

[0031] Figure 4 It is a structural diagram of a computer system suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0032] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0033] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0034] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of the present application can be applied, in which a data enhancement method for distinguishing the nature of crowd activities is applied.

[0035] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0036] Users can use terminal devices 101, 102, 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications can be installed on terminal devices 101, 102, 103, such as data processing applications, data visualization applications, web browser applications, etc.

[0037] Terminal devices 101, 102, and 103 can be hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules (for example, software or software modules used to provide distributed services), or they can be implemented as a single software or software module. No specific limitations are given here.

[0038] The server 105 may be a server that provides various services, such as a background information processing server that supports the mapping table data displayed on the terminal devices 101, 102, and 103. The background information processing server may process the obtained logical address and generate a processing result.

[0039] It should be noted that the method provided in the embodiment of the present application can be executed by the server 105 or by the terminal devices 101, 102, and 103. The corresponding device is generally set in the server 105 and can also be set in the terminal devices 101, 102, and 103.

[0040] It should be noted that the server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules (e.g., software or software modules for providing distributed services), or as a single software or software module. No specific limitations are given here.

[0041] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0042] According to an embodiment of the present application, a data enhancement method for distinguishing the nature of crowd activities is provided. Figure 2 FIG. 1 shows a flow chart of a data enhancement method for distinguishing the nature of crowd activities according to an embodiment of the present application. Figure 2 As shown, the method includes:

[0043] S201: Prepare a crowd activity training dataset and a pre-trained model for distinguishing the nature of crowd activities to generate a heat map. Models with strong effects on fine-grained image classification such as xception and Senet can be selected. Prepare a crowd activity training dataset defined as {(I i , Y i )|i=0, 1, ....N-1}, where, I i ∈R 3 ×W×H is a standard RGB image, i For image labels.

[0044] S202: Randomly extract a data pair from the crowd activity training dataset, use a pixel-level linear mixing enhancement strategy, and use linear combination to mix the image and label.

[0045] In a specific embodiment, a data pair {(I1, Y1), (I2, Y2)} is randomly extracted from a crowd activity training data set, and two parameters b1 and b2 are set to extract two pairs of proportional parameters (γ1, γ2), (γ3, γ4) from a Beta distribution Beta (b1, b2).

[0046] In a specific embodiment, a pixel-level linear mixing enhancement strategy is used, that is, a linear combination is used to mix the image and the label: M1 =γ1×T s (I1)+(1-γ1)×T s (I2); U a =γ1,U b =1-γ1;Y M1 =U a ×Y1+U b ×Y2; where I M1 is the mixed image, Y M1 is the corresponding mixed label, T s In order to meet the requirements of fusion morphological scale, random data enhancement functions of the same type (i.e., random rotation, translation, cropping, noise addition, scale scaling, quality transformation, etc.) are used. This can improve the overall generalization, introduce additional regularization effects, and have obvious benefits for crowd activity problems.

[0047] S203: Use the region-level affine stitching enhancement strategy to stitch images together through cut-and-paste operations and mix labels based on area ratios. Specifically, it is expressed as: Q a =1-γ2,Q b =γ2;Y M2 =Q a ×Y1+Q b ×Y2; where I M2 is the stitched image, Y M2 is the corresponding mixed label, Ts To meet the morphological scale requirements of the fusion, random data enhancement functions of the same type are used. This has the ability to integrate scene semantics, enrich the data set content, break the general empirical characteristics of crowd activities, and effectively alleviate scene overfitting.

[0048] S204: By strengthening the class gradient activation visualization strategy, extracting the output class activation heat map, performing image secondary hybrid enhancement and label fusion, and forming a secondary hybrid image enhancement dataset to expand the original dataset.

[0049] In a specific embodiment, an enhanced class gradient activation visualization strategy is used to extract the output class activation heat map. The specific method is as follows: in Represents the class activation heat map obtained for the Cth category, i, j represents the pixel coordinates, for L c Upsample it to make its size consistent with the input image, denoted as right Perform semantic mapping so that the sum of its pixels is 1. To activate the attention mask, is the adaptive coefficient, is the kth feature map.

[0050] In a specific embodiment, the final image mixing strategy is executed, that is, the image secondary mixing enhancement: in and are two binary masks, containing random box regions with area ratio γ3 and random box regions with area ratio γ4, TR θ As the conversion function, I M2 The final cropped area is converted to match I M1 The box area; the label fusion method is: Y Mix =C a ×Y M1 +C b ×Y M2 ,in, K I1 , K I2 It means that the semantic map of the corresponding class activation heat map is mapped so that the sum of its pixels is 1, which can be specifically expressed as C a , C b is the semantic weight of the secondary mixed label

[0051] In a specific embodiment, based on the above method, a secondary hybrid image enhancement dataset is formed to augment the original dataset. The augmentation ratio is 35% of the data generated by using the pixel-level linear hybrid enhancement strategy, 35% of the data generated by using the enhanced gradient activation visualization strategy, and 30% of the data generated by the secondary hybrid scheme.

[0052] This application's data augmentation method for identifying the nature of crowd activity proposes a novel crowd scene sample synthesis scheme, effectively and specifically expanding the relevant sample library. Both the expansion process and results can significantly positively impact crowd activity identification algorithms. This is the first to address the problem of identifying the nature of crowd activity from the perspective of scene overfitting, focusing on data augmentation and rationalizing and normalizing sample set distribution. This method addresses the root cause of the problem and achieves significant results. It is also adaptable to any framework and any algorithmic model.

[0053] Continue to refer Figure 3 , Figure 3 The framework diagram of a data enhancement system for distinguishing the nature of crowd activities according to an embodiment of the present application is shown. The system specifically includes a preparation unit 301, a pixel-level linear mixing enhancement unit 302, a region-level affine splicing enhancement unit 303 and a data set expansion unit 304. The preparation unit 301 is configured to prepare a crowd activity training data set and a pre-trained model for distinguishing the nature of crowd activities to generate a heat map; the pixel-level linear mixing enhancement unit 302 is configured to randomly extract a data pair from the crowd activity training data set, use a pixel-level linear mixing enhancement strategy, and use linear combination to mix images and labels; the region-level affine splicing enhancement unit 303 is configured to use a region-level affine splicing enhancement strategy to splice images through cut-and-paste operations and mix labels according to area ratios; the data set expansion unit 304 is configured to extract the output class activation heat map by strengthening the class gradient activation visualization strategy, perform image secondary mixing enhancement and label fusion, and form a secondary mixed image enhanced data set to expand the original data set.

[0054] Reference below Figure 4 , which shows a structural diagram of a computer system 400 suitable for implementing an electronic device of an embodiment of the present application. Figure 4 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0055] like Figure 4As shown, the computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage unit 408 into a random access memory (RAM) 403. Various programs and data required for the operation of the system 400 are also stored in the RAM 403. The CPU 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0056] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output section 407 including a liquid crystal display (LCD) and speakers; a storage section 408 including a hard disk; and a communication section 409 including a network interface card such as a LAN card or a modem. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. Removable media 411, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 410 as needed, so that computer programs read therefrom can be installed into the storage section 408 as needed.

[0057] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above-mentioned functions defined in the method of the present application are executed. It should be noted that the computer-readable storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0058] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0059] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0060] The modules described in the embodiments of the present application may be implemented in software or hardware.

[0061] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or it may exist independently and not be assembled into the electronic device. The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by the electronic device, the electronic device: prepares a crowd activity training data set and a pre-trained model for distinguishing the nature of crowd activities to generate a heat map; randomly extracts a data pair from the crowd activity training data set, uses a pixel-level linear mixing enhancement strategy, and uses linear combination to mix images and labels; uses a region-level affine splicing enhancement strategy to splice images through cut-and-paste operations, and mixes labels according to area ratios; extracts output class activation heat maps through an enhanced class gradient activation visualization strategy, performs image secondary mixing enhancement and label fusion, and forms a secondary mixed image enhancement data set to expand the original data set.

[0062] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A data enhancement method for distinguishing the nature of crowd activities, characterized by: include: S1: Prepare crowd activity training dataset and pre-trained model for crowd activity nature identification to generate heatmap; S2: randomly extract a data pair from the crowd activity training dataset, use a pixel-level linear mixing enhancement strategy, and use linear combination to mix the image and label; S3: Uses a region-level affine stitching enhancement strategy to stitch images together through cut-and-paste operations and mix labels based on area ratios; S4: By strengthening the class gradient activation visualization strategy, extracting the output class activation heat map, performing image secondary hybrid enhancement and label fusion, and forming a secondary hybrid image enhancement dataset to expand the original dataset.

2. The data enhancement method for distinguishing the nature of crowd activities according to claim 1 is characterized in that: The pre-training model includes xception or Senet, and the crowd activity training data set is defined as {(I i , Y i )|i=0, 1, ....N-1}, where, I i ∈R 3 ×W×H is a standard RGB image, i For image labels.

3. The data enhancement method for distinguishing the nature of crowd activities according to claim 2 is characterized in that: Specifically, S2 is as follows: randomly extract a data pair {(I1, Y1), (I2, Y2)} from the crowd activity training dataset, set two parameters b1 and b2 to extract two pairs of proportional parameters (γ1, γ2), (γ3, γ4) from a Beta distribution Beta (b1, b2); use linear combination to mix images and labels: I M1 =γ1×T s (I1)+(1-γ1)×T s (I2); U a =γ1,U b =1-γ1;Y M1 =U a ×Y1+U b ×Y2; where I M1 is the mixed image, Y M1 is the corresponding mixed label, T s It is a random data enhancement function of the same type that meets the fusion morphology scale requirements.

4. The data enhancement method for distinguishing the nature of crowd activities according to claim 3 is characterized in that: The S3 is specifically expressed as: Q a =1-γ2,Q b =γ2;Y M2 =Q a ×Y1+Q b ×Y2; where I M2 is the stitched image, Y M2 is the corresponding mixed label, T s In order to meet the fusion morphology scale requirements, random data enhancement functions of the same type are used.

5. The data enhancement method for distinguishing the nature of crowd activities according to claim 4 is characterized in that: In S4, the class gradient activation visualization strategy is strengthened to extract the output class activation heat map, which is specifically expressed as follows: in represents the class activation heat map obtained for the Cth category, i, j represents the pixel coordinates, To activate the attention mask, is the adaptive coefficient, is the kth feature map, for L c Upsample to make it the same size as the input image and obtain right Perform semantic graph mapping so that the sum of its pixels is 1.

6. The data enhancement method for distinguishing the nature of crowd activities according to claim 5, characterized in that: The image secondary hybrid enhancement in S4 is specifically as follows: in and are two binary masks, containing random box regions with area ratio γ3 and random box regions with area ratio γ4, TR θ As the conversion function, I M2 The final cropped area is converted to match I M1 The box area; the label fusion method is: Y Mix =C a ×Y M1 +C b ×Y M2 , where C a , C b is the semantic weight of the secondary mixed label.

7. The data enhancement method for distinguishing the nature of crowd activities according to claim 1 is characterized in that: The expansion ratio of the original data set in S4 is to generate 35% of data by the pixel-level linear mixing enhancement strategy, generate 35% of data by the region-level affine splicing enhancement strategy, and generate 30% of data by the image secondary mixing.

8. A computer-readable storage medium having one or more computer programs stored thereon, characterized in that: When the one or more computer programs are executed by a computer processor, the method according to any one of claims 1 to 7 is implemented.

9. A data enhancement system for distinguishing the nature of crowd activities, characterized by: The system comprises: Preparation unit: Configures a pre-trained model for preparing crowd activity training datasets and identifying the nature of crowd activity to generate heat maps; A pixel-level linear mixing enhancement unit is configured to randomly extract a data pair from the crowd activity training dataset, use a pixel-level linear mixing enhancement strategy, and use a linear combination to mix the image and the label; Region-level affine stitching enhancement unit: configured to use the region-level affine stitching enhancement strategy to stitch images through cut-and-paste operations and mix labels according to area ratios; Dataset expansion unit: It is configured to extract the output class activation heat map by strengthening the class gradient activation visualization strategy, perform image secondary hybrid enhancement and label fusion, and form a secondary hybrid image enhancement dataset to expand the original dataset.

10. The data enhancement system for distinguishing the nature of crowd activities according to claim 9, characterized in that: The pre-training model includes xception or Senet, and the crowd activity training data set is defined as {(I i , Y i )|i=0, 1, ....N-1}, where, I i ∈R 3 ×W×H is a standard RGB image, i For image labels.

11. The data enhancement system for distinguishing the nature of crowd activities according to claim 10, characterized in that: The pixel-level linear mixing enhancement unit is specifically configured to: randomly extract a data pair {(I1, Y1), (I2, γ2)} from the crowd activity training data set, set two parameters b1 and b2 to extract two pairs of proportional parameters (γ1, γ2), (γ3, γ4) from a Beta distribution Beta (b1, b2); use linear combination to mix images and labels: I M1 =γ1×T s (I1)+(1-γ1)×T s (I2); U a =γ1,U b =1-γ1;Y M1 =U a ×Y1+U b ×Y2; where I M1 is the mixed image, Y M1 is the corresponding mixed label, T s It is a random data enhancement function of the same type that meets the fusion morphology scale requirements.

12. The data enhancement system for distinguishing the nature of crowd activities according to claim 11, characterized in that: The region-level affine splicing enhancement unit is specifically expressed as: Q a =1-γ2,Q b =γ2;Y M2 =Q a ×Y1+Q b ×Y2; where I M2 is the stitched image, Y M2 is the corresponding mixed label, T s In order to meet the fusion morphology scale requirements, random data enhancement functions of the same type are used.

13. The data enhancement system for distinguishing the nature of crowd activities according to claim 12, characterized in that: The dataset expansion unit is specifically configured to extract the output class activation heat map by strengthening the class gradient activation visualization strategy, specifically expressed as: in represents the class activation heat map obtained for the Cth category, i, j represents the pixel coordinates, To activate the attention mask, is the adaptive coefficient, is the kth feature map, for L c Upsample to make it the same size as the input image and obtain right Perform semantic graph mapping so that the sum of its pixels is 1; The specific image secondary hybrid enhancement is: in and are two binary masks, containing random box regions with area ratio γ3 and random box regions with area ratio γ4, TR θ As the conversion function, I M2 The final cropped area is converted to match I M1 The box area; the label fusion method is: Y Mix =C a ×Y M1 +C b ×Y M2 , where C a , C b is the semantic weight of the secondary mixed label.

14. The data enhancement system for distinguishing the nature of crowd activities according to claim 9, characterized in that: The expansion ratio of the original data set is to generate 35% of data by the pixel-level linear mixing enhancement strategy, generate 35% of data by the region-level affine splicing enhancement strategy, and generate 30% of data by the image secondary mixing.

Images